Data Engineer

Home

People

Data Engineer

ID : 1212

Education level : Bachelor's degree

Work experience level : Expert- (more than year 7)

Work experience in total : Years

Job type : Online

Job time : Maandelijks

Last date of registration :

2022-12-20

Azure Data Engineer - Accenture Service Pvt Ltd-Mumbai (India) - (2015-2022)

Around 11+ years of IT experience as in and on Azure Cloud. Experience on Migrating SQL database to Azure data Lake, Azure data lake Analytics, Azure SQL Database, Data Bricks and Azure SQL Data warehouse and controlling and granting database access and Migrating On premise databases to Azure Data Lake store using Azure Data factory.
Domain knowledge and experience in Retail(e-Store), Banking (KYC and Account Transaction) Logistics(inventory) and social media (face book) and Instagram industries. Domain knowledge and experience in Retail, Banking, Logistics and Social Media industries.
Experience in Developing Spark applications using Spark - SQL in Databricks for data extraction, transformation and aggregation from multiple file formats for analysing & transforming the data to uncover insights into the customer usage patterns.
Good understanding of Spark Architecture including Spark Core, Spark SQL, Data Frames, Spark Streaming, Driver Node, Worker Node, Stages, Executors and Tasks.
Experience with MS SQL Server Integration Services (SSIS), T-SQL skills, stored procedures, triggers. Design and develop Spark applications using PySpark and Spark-SQL for data extraction, transformation and aggregation from multiple file
formats for analyzing & transforming the data to uncover insights into the customer usage patterns.
Good understanding of Big Data Hadoop and Yarn architecture along with various Hadoop Demons such as Job Tracker, Task Tracker, Name Node, Data Node, Resource/Cluster Manager, and Kafka (distributed stream-Processing).
Design and implement database solutions in Azure SQL Data Warehouse, Azure SQL.
Azure Data Factory (ADF), Integration Run Time (IR), File System Data Ingestion, Relational Data Ingestion
Created Azure SQL database, performed monitoring and restoring of Azure SQL database. Performed migration of Microsoft SQL server to Azure SQL database.
Have designed and developed ETL mapping for data collection from various data feeds using REST API. The data sources include feeds from mobile, social Facebook Query Language, You tube, Twitter, web and other partner feeds.
Generated Views, reports and dashboards (Tiles from multiple reports) using Power BI Used Dimensional Modelling Techniques to identify parameters necessary to create Dimensions, Fact and Aggregate tables.

Data Scientist - Accenture Service Pvt Ltd-Mumbai (India) -

Experience in Data Extraction, Data Modelling, Data Wrangling, Statistical Modeling, Data Mining, Machine Learning and Data Visualization.
Domain knowledge and experience in Retail, Banking, Logistics and Social Media industries.
Expertise in transforming business resources and requirements into manageable data formats and analytical models, designing algorithms, building models, developing data mining and reporting solutions that scale across a massive volume of structured and unstructured data.
Proficient in managing entire data science project life cycle and actively involved in all the phases of project life cycle including data acquisition, data cleaning, data engineering, features scaling, features engineering, statistical modeling, testing and validation and data visualization.
Proficient in Machine Learning algorithm and Predictive Modeling including Regression Models, Decision Tree, Random Forests, Sentiment Analysis, Naive Bayes Classifier, SVM, Ensemble Models.
Proficient in Statistical Methodologies including Hypothetical Testing, ANOVA, Time Series, Principal Component Analysis, Factor Analysis, Cluster Analysis, Discriminant Analysis.
Knowledge on time series analysis using AR, MA, ARIMA, GARCH and ARCH model.
Knowledge on Natural Language Processing (NLP) algorithm and Text Mining.
Generated Views, reports and dashboards (Tiles from multiple reports) using Power BI Used Dimensional Modeling Techniques to identify parameters necessary to create Dimensions, Fact and Aggregate tables.

Data Analyst - WNS Global Services-Mumbai (India)- (2011 -2015)

Performed a wide range of data mining and statistical analyses for the purpose of developing and implementing a data management system for the company.
Performed complex analysis of database performance and data quality using SQL. Maintained MS Excel spreadsheet for tracking and analysing the results of data.
Developed, implemented and managed a Python script to analyse and interpret the data from a variety of sources.
Developed an ETL Database Model to carry out new procedures while creating various database warehouse dimensions to accept ETL outputs.
Utilized Joins and sub-queries to simplify complex queries involving multiple tables while optimizing procedures and triggers in production.
Manage work to assigned due dates. Notify manager of any conflicts or other issues related to timely delivery of work.
Work closely with the development team to provide support for database objects (stored procedures/triggers/views) and performing code review.
Utilize various data visualization tools and techniques to present information.
Develop and publish a set of metrics capturing Data Quality trends for a week, month, and quarter to ensure our Data Quality programs are working effectively.
Develop active Data Quality monitoring and measurement, establishing and publishing KPIs and SLAs, Quality Report, RCA, Roster.

Azure Data Lake, Data factory, Azure Databricks, Azure SQL database, Azure Synapse Analytics, Azure Stream Analytics, Azure HDInsight, Azure SQL Datawarehouse.
Python (pandas, NumPy, Scipy, Scikit learn, TensorFlow, Keras)
SQL (MySQL, MS Access, PostgreSQL, Oracle and SQL Server)
Power BI, Excel and PowerPoint

Azure Data Lake, Data factory, Azure Databricks, Azure SQL database, Azure Synapse Analytics, Azure Stream Analytics, Azure HDInsight, Azure SQL Datawarehouse, Python (pandas, NumPy, Scipy, Scikit learn, TensorFlow, Keras), SQL (MySQL, MS Access, PostgreSQL, Oracle and SQL Server), Power BI, Excel, PowerPoint

Meer person

ID : 2042

Associate Software Engineer

Education level: Bachelor's degree

Work experience level: Intermediate- (2-4 year experience)

ID : 2041

System Engineer IT

Education level: Bachelor's degree

Work experience level: Experienced- (4-7 year experience)

ID : 2040

Associate Engineer

Education level: Bachelor's degree

Work experience level: Associate- (1-2 year experience)

ID : 2039

Intern

Education level: Bachelor's degree

Work experience level: Beginner- (internship- 1 year experience)

Data Engineer

Meer person

Mis geen enkele belangrijke kennisgeving houd jezelf update