• Anywhere

Job Description:

Any seasoned professional from our Data Engineering & Platform team would be a great fit. Open for suggestions.

  • Platform Architect having minimum 10 years of experience implementing Big Data solutions (including Large data sets in PBs- structured and unstructured) in large enterprise setting.
    Candidate has proficiency all tenets of being an enterprise solutions architect and should have fluency in.

    • Platform management experience at scale with Big data Hadoop components (HDFS, Yarn, Storm, HBASE, Hive, Spark, RabbitMQ, Ambari, Zoo Keeper, Grafana, Kafka, Zoo Keeper etc..,).
    • real-time ingestion/streaming options for large data set flows
    • with hands-on experience in designing, building & deploying big data solutions On-prem & Cloud (Azure) & Hybrid solutions.
    • Performance tuning (for eg:. Yarn is slow, Tez jobs are slow, Slow Data loading) & maintain platform integrity.
    • Industry best practices & recommendations review and roll out as appropriate
    • Security compliance.
      • Manage & maintain layered access through Authentication, Authorization, and Auditing.
      • Addition and maintenance of user access for both new and existing users.
      • Maintain & manage High Availability.
      • Manage permissions and roll over Ranger KMS keys.
      • Monitor the Automated Audit forwarding Job.
      • Audit Log Clean up as directed by security information and event management (SIEM) system.
      • Familiarity with open-source configuration management and deployment tools such as Puppet or Chef and Linux scripting.
    • Consumption of Big Data
      a.    Aggregation strategies
      b.    Micro service architecture
      c.    Subscription model
      d.    API  (Rest, Graphql)
      e.    Modeling for Insight/KPI factory
      f.    Product Development
    • Digital 360
      a.    Tooling- Jenkins, Dockers, Kubernetes, DevOps etc.
      b.    Process-
      c.    Methodologies- safe Agile
    • Working knowledge of Machine learning (good to have)
    • Telco Domain Knowledge (Preferable)
      a.    Big Data relative to Telecom (a. Understanding Key Data Element- EDR,CDR, LSR; b. Product Development for consumption from the Key data element)

Skills Required:

  • 10+ years of experience in data warehousing / big data environment, preferably in wireless industry and/or finance function
  • Experience building & optimizing data pipelines on cloud solutions such as AWS & Azure
  • Good understanding of system’s capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
  • Experience in managing the Distribution file systems.
  • Hadoop skills like HBase, Hive, Pig, Mahout, etc.
  • Experience in design, Development of Hadoop cluster, add /remove nodes, keeping track of jobs, monitoring critical parts of the cluster.
  • Good knowledge of Linux as Hadoop runs on Linux.
  • Knowledge of Troubleshooting Core Java Applications is a plus.
  • Extensive experience in Spark, Python, PySpark, Scala, SQL/Hive
  • Familiar with Spark MLlib, SparkSQL
  • Writing complex SQL statements and performing data discovery
  • Proven experience with data modeling, complex data structures, data processing, data quality, and data lifecycle
  • Experience in API design, SDK architecture, and mobile software lifecycle development practices.
  • Strong organizational and time management skills, ability to prioritize a large group of diverse projects simultaneously and deliver on multiple threads of work.
  • Experience working in agile environments on multiple projects simultaneously.
  • Excellent written and verbal communication skills.

To apply for this job email your details to zameer@cygnuspro.com