rikai.spark.types package¶
Submodules¶
rikai.spark.types.geometry module¶
Geometry User Defined Types in Spark
- class rikai.spark.types.geometry.Box2dType¶
Bases:
UserDefinedType
User defined type for the 2D bounding box.
- deserialize(datum: Row) rikai.types.geometry.Box2d ¶
Converts a SQL datum into a user-type object.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: rikai.types.geometry.Box2d)¶
Serialize a Box2d into a PySpark Row
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.geometry.Box3dType¶
Bases:
UserDefinedType
Spark UDT for
Box3d
class.- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.geometry.MaskType¶
Bases:
UserDefinedType
Spark UDT for 2-D mask
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.geometry.PointType¶
Bases:
UserDefinedType
Spark UDT for
rikai.types.geometry.Point
class.- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
rikai.spark.types.video module¶
- class rikai.spark.types.video.VideoStreamType¶
Bases:
UserDefinedType
VideoStreamType defines the Spark UserDefineType for a given video stream
- deserialize(datum) VideoStream ¶
Converts a SQL datum into a user-type object.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: VideoStream)¶
Serialize a VideoStream to a Spark Row
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.video.YouTubeVideoType¶
Bases:
UserDefinedType
YouTubeVideoType defines the Spark UserDefineType for a piece of YouTube video content (i.e., corresponds to a given youtube id but can have multiple streams)
- deserialize(datum) YouTubeVideo ¶
Converts a SQL datum into a user-type object.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: YouTubeVideo)¶
Serialize a YouTubeVideo
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
rikai.spark.types.vision module¶
- class rikai.spark.types.vision.ImageType¶
Bases:
UserDefinedType
ImageType defines the Spark UserDefineType for Image type
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
Module contents¶
Spark User Defined Types
- class rikai.spark.types.Box2dType¶
Bases:
UserDefinedType
User defined type for the 2D bounding box.
- deserialize(datum: Row) rikai.types.geometry.Box2d ¶
Converts a SQL datum into a user-type object.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: rikai.types.geometry.Box2d)¶
Serialize a Box2d into a PySpark Row
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.Box3dType¶
Bases:
UserDefinedType
Spark UDT for
Box3d
class.- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.ImageType¶
Bases:
UserDefinedType
ImageType defines the Spark UserDefineType for Image type
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.MaskType¶
Bases:
UserDefinedType
Spark UDT for 2-D mask
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.NDArrayType¶
Bases:
UserDefinedType
User define type for an arbitrary
numpy.ndarray
.This UDT serializes
numpy.ndarray
into bytes buffer.- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: ndarray)¶
Serialize an
numpy.ndarray
into Spark Row
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.PointType¶
Bases:
UserDefinedType
Spark UDT for
rikai.types.geometry.Point
class.- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.SegmentType¶
Bases:
UserDefinedType
Defines the Spark UDT for a video segment A video segment is defined by the starting frame number (start_fno) and ending frame number (end_fno). fno terminology is chosen to be consistent with the opencv library.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.VideoStreamType¶
Bases:
UserDefinedType
VideoStreamType defines the Spark UserDefineType for a given video stream
- deserialize(datum) VideoStream ¶
Converts a SQL datum into a user-type object.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: VideoStream)¶
Serialize a VideoStream to a Spark Row
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.
- class rikai.spark.types.YouTubeVideoType¶
Bases:
UserDefinedType
YouTubeVideoType defines the Spark UserDefineType for a piece of YouTube video content (i.e., corresponds to a given youtube id but can have multiple streams)
- deserialize(datum) YouTubeVideo ¶
Converts a SQL datum into a user-type object.
- classmethod scalaUDT() str ¶
The class name of the paired Scala UDT (could be ‘’, if there is no corresponding one).
- serialize(obj: YouTubeVideo)¶
Serialize a YouTubeVideo
- classmethod sqlType() StructType ¶
Underlying SQL storage type for this UDT.