Lead Data Engineer - Richland

Company: Pacific Northwest National Laboratory
Location: Richland
Posted on: January 27, 2023

Job Description:

**Overview**Join a group of 90 software engineers using the latest technologies to solve the hardest problems for our nation. We are seeking a lead data engineer to design, build, and deploy scalable data pipelines and analytics/machine learning solutions.**Critical Technologies**+ Programming & Scripting - Python, GO, Rust, Java, Scala+ Compute - IoT/ICS, Linux, Fargate/EC2, ECS/Docker, EKS/Kubernetes, EMR+ Development - Git/Gitlab, Agile, Atlassian, CDK, CI/CD, DevOps, IDE+ Data and Storage - S3, Athena, Postgres, Elasticsearch/OpenSearch, Dynamo, Redshift, MongoDB+ Pipelines - IAM, Cognito, Nifi, Airflow, Dagster, Spark, Lambda, Beats, Splunk+ Analytics - Dask, Numpy & Pandas, DataBricks, SageMaker, Tensorflow/Keras, PySpark**National Interest Project Examples**+ Detect and prevent smuggling of drugs and contraband at ports of entry [Link (https://www.pnnl.gov/sites/default/files/media/file/NII%20Capabilities 072621_0.pdf) ]+ Develop large data pipelines to thwart funding for terrorists, nuclear proliferators, drug cartels, and rogue leaders [Link (https://www.pnnl.gov/sites/default/files/media/file/PNNL\_Treasury\_AWS%20collab 1121.pdf) ]+ Applying big data solutions to national security problems [Link (https://www.pnnl.gov/news-media/science-front-line-ralph-perko) ]+ Applying image classification for nuclear forensics analysis [Link (https://www.pnnl.gov/sites/default/files/media/file/NSD\_1259\_FLYER\_SharkzorHighlights\_FINAL\_0.pdf) ]+ Detect and respond to advanced cyber threats with at-edge computing [Link (https://www.pnnl.gov/labobjectives/Cybersecurity.pdf) ]+ Develop capabilities for scalable geospatial analytics [Link (https://www.pnnl.gov/sites/default/files/media/file/GeoBOSS%20Open-Source Geospatial Analytics at Scale.pdf) ]+ Use remotely sensed imagery to identify and monitor the progression of wildfires [Link (https://www.pnnl.gov/news-media/disaster-response-and-mitigation-ai-world) ]+ Analyze the resiliency of the electric power grid to prevent large-scale outages [Link (https://www.pnnl.gov/building-and-grid-modeling-tools-and-capabilities) ]+ Optimize building efficiency using IOT and ICS data with automated demand-response markets [Link (https://volttron.org/) ]+ Model climate change and impacts to civilization [Link (https://im3.pnnl.gov/) ]+ Hunt for the existence of dark matter to understand the nature of the universe [Link (https://www.pnnl.gov/dark-matter) ]**Data Complexities**+ Volume - large, we work with terabytes and petabytes+ Variety - Images, audio, text, IoT, RF, GPS, edge sensors+ Velocity - Sub-second and lower frequency**How We Work**+ Diverse and flexible projects - Flexibility to choose and move between projects+ Agile development environment - Scrum meetings, standups, demos, and retrospectives+ Partners - Work with government, academic, industry, and other partners to solve problems+ Locations - Seattle, WA; Richland, WA; Washington, DC+ Team Sizes - Typically around 5-10 members, although projects can be more than 100 or just a few members+ Team Compositions - Our teams include cloud engineers, machine learning engineers, data scientists/domain experts, UI/UX designers, front-end developers, scrums masters, product owners, and most importantly, usersA day in the life of a data engineer at PNNL might involve exploring new scientific data and creating robust datasets. You will create data pipelines to store in large databases, feed to AI/ML and create new data analysis tools to tackle national level problems. You will work with Cloud engineers to deploy to AWS and Azure, ML researchers to develop production ready models, and analysts to extract new features and derive new insights. You will hit a sprint demo to show off your biweekly progress and then call it a day. All along you will know that your deployment answered a critical national security problem, something that might have been discussed on the evening news.Missing some of these skills or experiences? That's okay. If you have relevant technical expertise, are highly driven, and are very motivated to learn these technologies and tackle these domain problems, let's talk.\#LI-Hybrid**Responsibilities**+ Identify mission challenges and formulate engineering solutions methodically+ Embrace software engineering excellence and delivering quality results at scale+ Employ expertise with a high-level programming language such as Python+ Apply good design and innovative problem-solving skills to solve challenging technical problems+ Stay current about data management and database technology developments+ Initiate personal direction and goals+ Possess interest and/or experience mentoring junior scientists and engineers+ Demonstrate outstanding verbal and written communication skills and the ability to work in a collaborative environment+ Be passionate and self-motivated with good time management skills**Qualifications**Minimum Qualifications:+ BS/BA and 7+ years of relevant work experience -OR-+ MS/MA and 5+ years of relevant work experience -OR-+ PhD with 3+ year of relevant experiencePreferred Qualifications:+ Degree in computer science, software engineering, or related field+ 5-7 years of experience in designing or deploying large-scale and high-performance ETL pipelines and analytics+ 7+ years Python or other software development experience+ Strong understanding of relational databases, NoSQL databases, and query authoring+ 5+ years of experience with SQL and Data Modeling+ Strong understanding of software engineering and data management best practices+ Strong cloud architecture and implementation experience+ Cloud and database certifications are a plus+ Familiar with machine learning algorithms with hands-on experience in machine learning pipeline development is a plus.+ Active Federal Q Clearance and ability to maintain such clearance**Hazardous Working Conditions/Environment**Not applicable.**Additional Information**This position requires the ability to obtain and maintain a federal security clearance (Q/SCI).Requirements:+ U.S. Citizenship+ Background Investigation: Applicants selected will be subject to a Federal background investigation and must meet eligibility requirements for access to classified matter in accordance with 10 CFR 710, Appendix B.+ Drug Testing: All Security Clearance positions are Testing Designated Positions, which means that the candidate selected is subject to pre-employment and random drug testing. In addition, applicants must be able to demonstrate non-use of illegal drugs, including marijuana, for the 12 consecutive months preceding completion of the requisite Questionnaire for National Security Positions (QNSP).Note: Applicants will be considered ineligible for security clearance processing by the U.S. Department of Energy until non-use of illegal drugs, including marijuana, for 12 months can be demonstrated.referral eligible**Testing Designated Position**This position is a Testing Designated Position (TDP). The candidate selected for this position will be subject to pre-employment and random drug testing for illegal drugs, including marijuana, consistent with the Controlled Substances Act and the PNNL Workplace Substance Abuse Program.**Commitment to Excellence, Diversity, Equity, Inclusion, and Equal Employment Opportunity**Our laboratory is committed to a diverse and inclusive work environment dedicated to solving critical challenges in fundamental sciences, national security, and energy resiliency. We are proud to be an Equal Employment Opportunity and Affirmative Action employer. In support of this commitment, we encourage people of all racial/ethnic identities, women, veterans, and individuals with disabilities to apply for employment.Pacific Northwest National Laboratory considers all applicants for employment without regard to race, religion, color, sex (including pregnancy, sexual orientation, and gender identity), national origin, age, disability, genetic information (including family medical history), protected veteran status, and any other status or characteristic protected by federal, state, and/or local laws.We are committed to providing reasonable accommodations for individuals with disabilities and disabled veterans in our job application procedures and in employment. If you need assistance or an accommodation due to a disability, contact us at careers@pnnl.gov .**Drug Free Workplace**PNNL is committed to a drug-free workplace supported by Workplace Substance Abuse Program (WSAP) and complies with federal laws prohibiting the possession and use of illegal drugs.**Mandatory Requirements**Battelle requires employees to have a COVID-19 vaccine as a condition of employment, subject to accommodation. Applicants are required to disclose their vaccination status following a conditional offer of employment and must attest to being fully vaccinated with a Center for Disease Control (CDC)-approved COVID-19 vaccination or provide documentation of need for medical or religious exemption from the COVID-19 vaccination requirement.Please be aware that the Department of Energy (DOE) prohibits DOE employees and contractors from having any affiliation with the foreign government of a country DOE has identified as a "country of risk" without explicit approval by DOE and Battelle. If you are offered a position at PNNL and currently have any affiliation with the government of one of these countries, you will be required to disclose this information and recuse yourself of that affiliation or receive approval from DOE and Battelle prior to your first day of employment.**Rockstar Rewards**Employees and their families are offered medical insurance, dental insurance, vision insurance, health savings account, flexible spending accounts, basic life insurance, disability insurance*, employee assistance program, business travel insurance, tuition assistance, supplemental parental bonding leave**, surrogacy and adoption assistance, and fertility support. Employees are automatically enrolled in our company funded pension plan* and may enroll in our 401k savings plan. Employees may accrue up to 120 vacation hours per year and may receive ten paid holidays per year.* Research Associates excluded.**Once eligibility requirements are met.Click Here For Rockstar Rewards (https://careers.pnnl.gov/rockstar-rewards)

Keywords: Pacific Northwest National Laboratory, Richland , Lead Data Engineer - Richland, Other , Richland, Washington