Responsibilities
Design, develop, and maintain Python and PySpark solutions, including data analysis programs and a custom rules-generation framework. Gather data from HBase, apply business transformations with Spark DataFrames and RDDs, perform Hive read/write operations, and develop and optimize back-end services and APIs.
Requirements
The posting requires strong proficiency in Python and Spark, with PySpark listed as a mandatory skill. It also describes experience developing Python and PySpark programs, building a custom rules framework, and working with HBase, Spark DataFrames or RDDs, and Hive.
