Any seasoned professional from our Data Engineering & Platform team would be a great fit. Open for suggestions.
- Platform Architect having minimum 10 years of experience implementing Big Data solutions (including Large data sets in PBs- structured and unstructured) in large enterprise setting.
Candidate has proficiency all tenets of being an enterprise solutions architect and should have fluency in.
- Platform management experience at scale with Big data Hadoop components (HDFS, Yarn, Storm, HBASE, Hive, Spark, RabbitMQ, Ambari, Zoo Keeper, Grafana, Kafka, Zoo Keeper etc..,).
- real-time ingestion/streaming options for large data set flows
- with hands-on experience in designing, building & deploying big data solutions On-prem & Cloud (Azure) & Hybrid solutions.
- Performance tuning (for eg:. Yarn is slow, Tez jobs are slow, Slow Data loading) & maintain platform integrity.
- Industry best practices & recommendations review and roll out as appropriate
- Security compliance.
- Manage & maintain layered access through Authentication, Authorization, and Auditing.
- Addition and maintenance of user access for both new and existing users.
- Maintain & manage High Availability.
- Manage permissions and roll over Ranger KMS keys.
- Monitor the Automated Audit forwarding Job.
- Audit Log Clean up as directed by security information and event management (SIEM) system.
- Familiarity with open-source configuration management and deployment tools such as Puppet or Chef and Linux scripting.
- Consumption of Big Data
a. Aggregation strategies
b. Micro service architecture
c. Subscription model
d. API (Rest, Graphql)
e. Modeling for Insight/KPI factory
f. Product Development
- Digital 360
a. Tooling- Jenkins, Dockers, Kubernetes, DevOps etc.
c. Methodologies- safe Agile
- Working knowledge of Machine learning (good to have)
- Telco Domain Knowledge (Preferable)
a. Big Data relative to Telecom (a. Understanding Key Data Element- EDR,CDR, LSR; b. Product Development for consumption from the Key data element)
- 10+ years of experience in data warehousing / big data environment, preferably in wireless industry and/or finance function
- Experience building & optimizing data pipelines on cloud solutions such as AWS & Azure
- Good understanding of system’s capacity, bottlenecks, basics of memory, CPU, OS, storage, and networks.
- Experience in managing the Distribution file systems.
- Hadoop skills like HBase, Hive, Pig, Mahout, etc.
- Experience in design, Development of Hadoop cluster, add /remove nodes, keeping track of jobs, monitoring critical parts of the cluster.
- Good knowledge of Linux as Hadoop runs on Linux.
- Knowledge of Troubleshooting Core Java Applications is a plus.
- Extensive experience in Spark, Python, PySpark, Scala, SQL/Hive
- Familiar with Spark MLlib, SparkSQL
- Writing complex SQL statements and performing data discovery
- Proven experience with data modeling, complex data structures, data processing, data quality, and data lifecycle
- Experience in API design, SDK architecture, and mobile software lifecycle development practices.
- Strong organizational and time management skills, ability to prioritize a large group of diverse projects simultaneously and deliver on multiple threads of work.
- Experience working in agile environments on multiple projects simultaneously.
- Excellent written and verbal communication skills.
To apply for this job email your details to email@example.com