This Research Computing Data Engineer position provides expert support for the Vermont Advanced Computing Center’s (VACC) research computing community, focusing on large-scale data pipelines, analysis, and visualization. The RC Data Engineer develops and implements code and pipelines that extract, transform, and load data into high performance systems such as sharded MongoDB clusters. They also help support development of web-based visualization services using JavaScript libraries such as D3. This position will help identify new corpora to meet research objectives, work with researchers to determine how best to store that data to meet computational goals, and build tools to analyze data. The Data Engineer will coordinate with faculty, researchers, and students to develop new and enhance existing web services for data-driven visualization. The Data Engineer will develop APIs to enhance ability for people to interact with the data in novel ways.

This position joins a team that supports the HPC hardware, cluster software, and researcher software in the VACC. By providing data analysis tools and management, developing code, and supporting web visualizations, this position directly supports University of Vermont’s goal of enhancing IT resources available to the UVM research community. These efforts support UVM researchers, and enhance outreach and community engagement goals.

Experience developing software in languages such as Python and JavaScript is critical for this position.

Minimum Qualifications (or equivalent combination of education and experience)

Combination of education and/or applicable experience equivalent to a doctorate degree in a STEM related field, focused on software development and large data software and systems.

Programming experience, ideally with multiple languages such as Python, C, C++, Perl, bash. Experience with software version control such as Git.

Experience developing APIs for data access.

Experience administering non-relational databases such as MongoDB. Experience with development tools such as PyMongo.

Experience with HPC architectures, and appropriate use of CPUs, memory, storage, interconnects, and GPU architectures.

Enthusiasm for learning and evaluating new technologies and ability to determine if appropriate for adoption.

Expertise installing software on Linux-based systems, often from source.

Strong technical and problem-solving skills. Ability to think creatively about technical challenges.

Excellent customer service ethic, effective communication skills and collaborative approach to teamwork.

Desirable Qualifications

Experience with JavaScript frameworks for data-driven visualization, such as D3. Experience with Node.js.

Experience building processes supporting data transformation, data structures, metadata, dependency and workload management.

Experience in debugging and tuning of software applications on HPC clusters, experience with common scientific libraries and applications.

Experience creating, using, and updating automated testing.

Experience working at an academic institution.

Familiarity with Linux systems engineering and understanding of Linux systems deployment.