Senior Data Engineer

Senior Data Engineer - Team *instinctools About company *instinctools Global *instinctools India *instinctools Poland *instinctools Kazakhstan Career Vacancies Team Blog Events Contact Us 26 Vacancies All jobs Senior Data Engineer Home Vacancies Senior Data Engineer *instinctools is a software development company that provides custom software solutions for businesses of all sizes. Our team works closely with clients to understand their specific needs and provide personalized solutions that meet their business requirements. *instinctools is looking for a Senior Data Engineer for one of our clients. Our Client is one of the TOP-5 global management consulting firms considered to be among the most prestigious ones in the world. Hundreds of customers from Fortune-500 , including the largest global financial institutions, the world’s top media companies, technology companies and federal government agencies rely on our Client’s proven platform and services. Project is a dynamic solution empowering companies to optimize promotional activities for maximum impact. It collects and validates data, analyzes promotion effectiveness, plans calendars, and integrates seamlessly with existing systems. The tool enhances vendor collaboration, negotiates better deals, and employs machine learning to optimize promotional plans, enabling companies to make informed decisions and maximize return on investment. Stack on the project: Databricks, SQL, Spark, Python Delta Lake (Delta Live Tables, Unity Catalog, Delta Sharing) Airflow, Azure Data Factory CI/CD pipelines (Azure DevOps / GitHub Actions or equivalent) Cloud Platforms (Azure preferred) BI and analytics frameworks (DBT) Project duration: 12 months (with possible extension) Location: Noida (office) Tasks: Design, build, and optimize scalable and reusable data pipelines on Databricks, SQL, and Spark Orchestrate automated ETL workflows using Airflow, ADF, and CI/CD pipelines Implement and manage scalable data processing systemsensuring performance, reliability, and cost-effectiveness Ensure data quality, consistency, and compliance with governance/security standards Collaborate with stakeholders to clarify requirements and translate them into technical solutions Continuously evaluate and implement new tools/technologies to improve pipeline automation, monitoring, and efficiency Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Our expectations of the ideal candidate: Bachelor’s or Master’s degree in Computer Science, Engineering, or related field 6+ years of experience in Data Engineering with strong big data expertise Deep expertise in Databricks, Spark, and SQL (including Spark SQL) Strong data modeling and ETL experience with large-scale datasets Hands-on experience orchestrating data pipelines with Airflow and CI/CD Proven experience with Delta Live Tables, Unity Catalog, Delta Sharing, and SQL Warehouse (server & serverless) Proficiency with Python for data engineering workflows Familiarity with analytics engineering and DBT Experience with data governance, compliance, and security in a cloud environment Strong problem-solving ability, requirement clarification, and communication skills Experience adopting software engineering best practices (testing, modularization, refactoring, automation) Ability to optimize data systems for performance and cost efficiency in cloud environments Exposure to AI/ML frameworks for development efficiency Demonstrated ability to lead projects independently and mentor junior engineers Soft Skills: Strong self-management: ability to work independently and meet deadlines Proactive and ownership-driven mindset Effective and clear communication, including in distributed teams Collaborative, open to feedback, and focused on shared goals Strong problem-solving skills, especially in ambiguous contexts High attention to detail and code quality Adaptability to shifting priorities and new technologies English: Upper-Intermediate or higher Working time zone: IST+ EU overlap Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data Send Data We offer: flexible working time (from Indian location) professional and ambitious team learning opportunities, seminars and conferences and time for exploring new technologies full remote work co-funding for language courses (English) Important Notice Regarding Your Personal Data Security At *instinctools, we take the security of your personal data very seriously. Please be informed that all official communication, offers, and correspondence from *instinctools are conducted only through our verified company resources, including: Official Email Domain: @instinctools.com Company Website: https://www.instinctools.com Important Reminder: *instinctools will NEVER ask for sensitive personal or financial information, including: – Bank account details – Credit/debit card numbers – Passwords or PINs If you receive any suspicious requests claiming to be from *instinctools via unofficial channels (e.g., personal emails, social media DMs, or phone calls), please: – Do not respond or share any information. – Report it immediately to compliance@instinctools.com or your official contact. Your trust and security are our top priorities. For any questions, always verify through our official channels. Thank you for your attention and cooperation. share Margarita Lukyanenka Recruiter We will be glad to see you in our team! * Full Name * E-mail * Job Vacancy * Location (country, city) Years of experience on a similar role/position Additional Information (LinkedIn, Github, etc.) Attach CV * * I confirm my consent to the processing of personal data and accept the terms of the Privacy Policy . Submit job application Developers *instinctools is a developer of IT solutions for Western Europe. We have over 350 projects in our portfolio. The spheres are different – from healthcare and medicine, training and education, to digital advertising, economics and IT. For the software implementation and maintenance across such a number of projects, competent developers are important. We use a modern technology stack, the key of which are: Javascript, ReactJS, Angular, Node.js, Golang, Python, Java, iOS, Android, Blockchain, AWS. Join the *instinctools team! *instinctools Poland We have had a representative office in Poland (Warsaw) since 2021. Our Polish team consists of highly qualified and experienced specialists – most of them are Middle to Senior level Developers. Right now we’re on the lookout for bright and innovative minds to join our professional team! Copyright © 2000 − 2024 — *instinctools / Privacy policy / FAQ Linkedin We are using cookies to give you the best experience on our website. You can find out more about which cookies we are using or switch them off in settings . Close GDPR Cookie Banner Accept All Reject All Settings Close GDPR Cookie Banner Change cookie settings Close GDPR Cookie Settings About this Policy Strictly Necessary Cookies Analytics cookies Cookie Policy About this Policy This Cookie Policy ("Policy") explains how we use cookies and other similar technologies to recognise you when you visit our website https://www.instinctools.eu/ ("Website"). It explains what these technologies are and why we use them, as well as your rights to control our use of them. The Website owned by INSTINCTOOLS PL SP. Z O. O. (referred to as “we”, “us”, or “our”). Please take the time to read this Policy carefully. If you have any questions or comments, please contact us via email at privacy@instinctools.eu . What are cookies? Cookies are small data files that are placed on your computer or mobile device when you visit a website. Cookies are used by website owners in order to make their websites work, or to work more efficiently, as well as to provide reporting information. Cookies have many different features, such as allowing you to navigate between pages efficiently, remembering your preferences, and generally improving your user experience. They can also help ensure that the advertisements you see online are more relevant to you and your interests. Cookies set by the website owner are called "first party cookies". Cookies set by other parties other than the website owner are called "third party cookies". Third party cookies enable third party features or functionality to be provided on or through the website (e.g. like advertising, interactive content and analytics). The parties that set these third party cookies can recognise your computer or mobile device both when it visits the websites in question and also when it visits certain other websites. Strictly Necessary Cookies These cookies are necessary for the website to function and cannot be switched off through the cookie-banner. They are usually only set to provide website services properly. You can set your browser to block or alert you about these cookies, but this may cause parts of the website to not work properly. Also we record and keep your language and region preferences and privacy choices once you have made them. Cookie name Host Description Duration pll_language www.instinctools.eu Used to set the language of web pages 24 hours Enable or Disable Cookies Analytics cookies Using these cookies we can count visits and traffic sources to be able to measure and improve the performance of our website. They help us to know which pages are the most and least popular and see how visitors move around the website. If these cookies are disabled, we will not be able to monitor the website’s performance. Cookie name Host Description Duration _ga Google Used to distinguish user 2 years _ga_M6HW5W7L8G Google Used to persist session state 2 years Enable or Disable Cookies Cookie Policy More information about our Cookie Policy. How can I control cookies? You have the right to decide whether to accept or reject specific types of cookies (except of strictly necessary cookies). You can exercise your cookie preferences on the cookie banner. You can set or amend your web browser controls to accept or refuse cookies. If you choose to reject cookies, you may still use our Website though your access to some functionality and areas of our Website may be restricted. As means by which you can refuse cookies through your web browser controls vary from browser to browser, you should visit your browser's help menu for more information. Updates to this policy We may update this Policy from time to time in response to legal, technical or business changes and developments. You can see when this Policy was last updated by checking the “last updated” date displayed at the top of this Policy. Enable All Reject All Save Changes