Please note that this role requires security clearance at SC level. You must be SC cleared to be considered for the role.
Tasks and Responsibilities
Design:
Define Data Platform technical architecture by analyzing the requirements.
Define technical design for ingestion, visualization and integration
Analyze the data mapping and data models and come up with ingestion level data mapping
Coordinate and support dev and testing team
Data migration design
Define and execute data migration
Implement POC with limited developer support
Provide expert advice to product teams on best way to use existing services and utilities from the repository
Design and lead the Azure Devops implementation for data platform
Define logical and physical data models
Lead:
Lead the development team in best practice Devops processes:
Engage in and improve the whole lifecycle of services—from inception and design, through deployment, operation and refinement
Coaching and mentoring the product team engineers and developers on DevOps practices
Collaborate with the shared platform teams/architecture team and a wider community internally and externally
Operational reliability:
Act as the technical point of contact on the platform for the cloud service providers (e.g. Azure) to ensure the Platform is operational and is able to meet the service levels of the products using the platform
Support services before they go live through activities such as system design consulting, developing software platforms and frameworks, capacity planning and launch reviews
Maintain services once they are live by measuring and monitoring availability, latency and overall system health
Scale systems sustainably through mechanisms like automation, and evolve systems by pushing for changes that improve reliability and velocity
Lead incident response and resolve issues on the platform
Continuously improving our security, failover, resilience and disaster recovery mechanisms
To work with solution architect , Lead Developers on the adoption of new technologies into the estate, ensuring that these are shared with other areas and that they align with organization standards etc
To support the team in identifying issues with code / deliverables
To champion and drive through alerting and monitoring requirements for the platform
Identify and execute pro-active actions to ensure continued stability and performance of the platform
Our ideal candidate
Strong experience in Designing and delivering Azure based data platform solutions ,technologies including
Azure data bricks,Azure Synapse Analytics
Azure MS SQL managed service/Azure postgreSQL
Azure ADF and Data bricks, Azure Functions,App Service ,Logic app,AKS,Azure app servie,Webapp
Good knowledge in real time streaming applications preferably with experienceKafka Real time messaging or Azure Stream Analytics / Event Hub.
Spark processing and performance tuning
File formats partitioning for e.g. Parquet,JSON,XML,CSV
Azure Devops,GIT hub actions
- Hands on experience in at least one of Python with knowledge of the others
- Experience in Data modeling
Experience of synchronous and asynchronous interface approaches
Experience of designing and developing systems using micro services architectural patterns
DevOps experience in implementing development, testing, release and deployment processes using DevOps processes.
Knowledge in data modeling (3NF/Dimensional modeling/Data Vault2)
Work experience in agile delivery
Able to provide comprehensive documentation
Able to set and manage realistic expectations for timescales, costs, benefits and measures for success
Able to lead, undertake and interpret technical analysis
Autonomy
Works under broad direction. Work is often self-initiated. Is fully responsible for meeting allocated technical and/or project/supervisory objectives. Establishes milestones and has a significant role in the assignment of tasks and/or responsibilities.
Influence
Influences organization, customers, suppliers, partners and peers on the contribution of own specialism. Builds appropriate and effective business relationships. Makes decisions, which affect the success of assigned work, i.e. results, deadlines and budget. Has significant influence over the allocation and management of resources appropriate to given assignments.
Complexity
performs an extensive range and variety of complex technical and/or professional work activities. Undertakes work, which requires the application of fundamental principles in a wide and often unpredictable range of contexts. Understands the relationship between own specialism and wider customer/organizational requirements.
Nice to Have
- Experience with integration and implementation of data cataloging tool like Azure Purview,Unity Catalog
- Experience in implementing and integrating visualizations tools like Power BI/Tableau etc
- Snowflake knowledge
