Data Engineer

HKS, Dallas

1. • Designed and built scalable data pipelines using Azure Data Factory and Azure Databricks, automating data workflows and ensuring seamless integration across various data sources for architecture and planning projects.
2. • Optimized ETL processes with Apache Spark on Azure Databricks, significantly improving data processing speed and efficiency for large datasets.
3. • Developed and maintained robust ETL processes for on-premises systems, optimizing data extraction, transformation, and loading operations.
4. • Engineered and managed scalable Azure cloud pipelines to streamline data integration and processing, enhancing overall data flow efficiency.
5. • Designed and implemented an XML API for seamless outbound integration, resulting in a 30% reduction in data exchange errors and improved data consistency.
6. • Led the migration of outbound integrations from on-premises to Azure cloud, achieving a 40% improvement in data processing speed and reducing operational costs.
7. • Provided production support, addressing and resolving data-related issues swiftly to ensure minimal disruption to business operations.
8. • Developed and enhanced extract, transform, and load (ETL) processes using Azure Synapse Analytics, ensuring accurate data analysis and supporting strategic project decisions.
9. • Improved data movement activities by utilizing Azure Event Hubs for real-time data streaming, maintaining data integrity across complex workflows.
10. • Collaborated with vendors to implement EDI solutions, integrating diverse data sources with Azure Logic Apps for seamless data exchanges.
11. • Led the selection of ETL tools and contributed to the design and architecture of data pipelines on the Azure platform, aligning with HKS’s data strategy and project goals.
12. • Developed semantic models using Azure Analysis Services to support reporting and visualization, providing actionable insights to project teams.

Data Engineer

Texas Health and Human Services Commission

1. • Mastered a wide array of Azure technologies, including HDInsight, Databricks, Data Lake, Blob Storage, Data Factory, Synapse Analytics, Azure SQL Database, and SQL Data Warehouse, to streamline cloud operations and data management.
2. • Designed and automated custom input adapters using Spark, Sqoop, and Oozie, significantly enhancing data ingestion from RDBMS to Azure Data Lake, showcasing a blend of innovation and efficiency in data handling.
3. • Played a pivotal role in the creation of automated workflows for daily incremental data loads, facilitating seamless data transition from traditional RDBMS systems to more scalable data lake solutions, thus optimizing data availability and reliability.
4. • Executed comprehensive ETL (Extract, Transform, Load) processes, utilizing Azure Data Factory, T-SQL, Spark SQL, and U-SQL for Azure Data Lake Analytics, ensuring efficient and scalable data movement and transformation strategies.
5. • Engineered and deployed data pipelines in Azure Data Factory, targeting enhanced ETL processes from diverse sources such as Azure SQL, Blob storage, and Azure SQL Data Warehouse, thereby ensuring data integrity and accessibility.
6. • Crafted and deployed JSON scripts in Azure Data Factory, leveraging SQL Activity for efficient data processing and integration, highlighting technical proficiency and advanced deployment capabilities.
7. • Demonstrated deep understanding of data ingestion and orchestration using Airflow Operators and relevant Python libraries, enhancing data flow and operational efficiency.
8. • Analyzed SQL scripts and implemented solutions using PySpark, focusing on optimizing data processing and analytics workflows.
9. • Extensively used Databricks notebooks for interactive analytics, employing Spark APIs to facilitate advanced data analysis and insights.
10. • Integrated data across various sources including MongoDB, MS SQL, and cloud platforms, using Azure Data Factory, SQL API, and Mongo API, to ensure a unified data ecosystem.
11. • Enabled efficient data visualization and reporting through interactive Power BI dashboards and reports, providing actionable insights to decision-makers.
12. • Spearheaded the Continuous Integration and Continuous Deployment (CI/CD) of applications into Azure Cloud, enhancing operational agility and system reliability.

The University of Texas at Dallas

Mathematics