United States
Suitability/Public Trust
Fully remote
Information Technology
Overview
GovCIO is currently hiring a Software Development Engineer in Test (SDET), Test Engineer, to join cross-functional product teams building and modernizing federal digital services on AWS GovCloud.
This position is in the Continental United States (CONUS) and remote.
Job Description
A GovCIO SDET works in a cross-functional product team where everyone owns quality. The SDET plans testing for each story, builds and maintains automated tests, and reports what is covered and what is not. The role is hands-on, automation-first, and deadline-driven. Story testing starts and finishes inside the sprint; the SDET also creates the longer tests that span sprints, such as functional, regression, end-to-end, integration, performance, and load. A GovCIO SDET is not a script executor, not a click-through tester, and not the last gate before release. Quality starts at refinement: the SDET makes acceptance criteria testable, works from user stories without waiting for a BRD or FRD, and audits the tests developers write. The SDET knows what to regression test and where to look when something breaks, and can evaluate non-deterministic output using measurable thresholds that separate model variance from real defects.
Responsibilities
General
Designs, develops, and implements testing methods and equipment to ensure that the product will perform to specifications. Plans and arranges the labor, schedules, and equipment required for testing and evaluating the product. Designs test cases for test plans and creates test scenarios in which tests are carried out. Compiles data, defines required changes and reports defects and problems that occurred during the test process. Implements and participates in the walkthrough of the product. Automated testing may be used for leveling based on hiring manager discretion.
- Consults with the development and/or requirements team to coordinate and test applications.
- Performs analysis, diagnostics and preliminary evaluations of products. Develop testing procedures.
- Develops test plans, scripts, scenarios, functional tests, regression tests, and deployment tests.
- Implements the testing processes and ensures all test activities follow those processes. Produces reports on each test activity and tracks test metrics.
- Monitors for continual test process improvement through failure analysis, throughput analysis, yield analysis, and report to management periodically.
Specific
- Work as a full member of a Scrum team. Take part in sprint planning, backlog refinement, and design walkthroughs. Make user stories and acceptance criteria testable. Write your own stories and tasks.
- Design and run positive and negative tests: functional, integration, regression, API, and end-to-end. Find or create the test data. Pace the work so testing finishes inside the sprint.
- Build and maintain the automated test suites. Use Cypress and/or Playwright for the front end, Postman or scripted tests for REST APIs, and Locust for load and performance. Put the suites in the CI/CD pipeline with the DevOps engineers. Build and manage test environments on AWS GovCloud.
- Use Amazon S3 to check test data and artifacts. Use Amazon CloudWatch and Logs Insights queries for logs, metrics, and troubleshooting during test runs.
- Own regression testing for the product. Know how the system fits together. Decide what must be re-tested and in what order. Keep the regression suite current as the product changes.
- Audit the unit, contract, integration, and end-to-end tests the developers write. Work with them to improve coverage and quality.
- Check data in PostgreSQL and other data stores with SQL. Check JSON, XML, and file content with scripts.
- Test machine learning and deep learning features, especially the large language models and vision models behind OCR and intelligent document processing (IDP). Build and maintain ground-truth sets. Measure accuracy at the character, field, and document level and as a straight-through rate. Set threshold-based acceptance criteria and be able to say where each number came from. Check whether confidence scores predict correctness.
- Evaluate generative AI features. Run eval harnesses against versioned prompts. Test schema conformance and answer correctness as separate questions. Catch values the model made up that are not in the source document.
- Automate eval runs in CI so regressions show up before UAT.
- Keep the Requirements Traceability Matrix, defect records, test results, and coverage and defect summary reports current in Jira, Xray, and Confluence.
- Perform root cause analysis and defect isolation with the developers.
- Post short status in Jira every day, in plain English, that shows how the work contributes to the sprint goal and where it fits in the big picture. Every engineer on the team is expected to address anything that puts product quality, the sprint goal, or the release date at risk.
- Support UAT and Section 508 staging reviews. Support Authority to Operate (ATO), security, and compliance work. That includes PHI/PII safeguards in test data and evaluations.
- Earn and maintain AWS certifications as part of ongoing professional development.
Qualifications
Required Skills and Experience
- Bachelor's with 2-5 years (or commensurate experience)
- The candidate must complete the paperwork promptly and pass the background check.
- Must be a U.S. citizen or permanent resident.
- Must reside and work within the Continental United States (CONUS).
- Test automation (minimum): Cypress or Playwright plus at least one other framework. Selenium alone is not enough. A candidate who has not used Cypress must explain the architectural difference between Cypress and other frameworks such as Selenium or Playwright and must have the front-end skills to explain the implementation difference.
- Automated REST API testing with Postman or scripted tests.
- SQL for data validation. Python for test scripting and for working with data at scale.
- Able to test systems built on machine learning and deep learning, especially large language models and vision models. Understands non-determinism, ground truth, and threshold-based acceptance.
- ML: training versus inference, overfitting, class imbalance, and gaps between lab results and production results.
- Experience testing in AWS environments, including S3 and CloudWatch.
- Agile testing in Scrum: story refinement, testability of acceptance criteria, definition of done, and pacing tests to close the sprint.
- Jira, Xray, and Confluence for test management and reporting.
- Full range of test techniques: positive and negative cases; white, black, and gray box; boundary value analysis and equivalence partitioning; use case testing; root cause analysis.
- Clear written and verbal communication. Gives specific technical answers, not generalities. Says plainly what they do not know. Writes short daily updates in Jira and writes their own user stories and tasks. Comfortable on voice and video calls with the team.
- Takes ownership and initiative without hand-holding and gets ahead of problems. Emotionally stable, honest, and professional, especially when troubleshooting under pressure.
- Can explain how they would pick testing tools and approaches for a given system, not only how to run the tools they already know.
- Any certifications listed on the resume must be current and verifiable.
- Keeps up with senior engineers and architects. Learns new tools quickly.
- Willing to earn the AWS Solutions Architect Associate certification within the first year if not already held.
Clearance Required: Public Trust.
Preferred Skills and Experience
- AWS certification (Solutions Architect Associate or higher) strongly preferred.
- Front-end development experience (JavaScript, single-page applications) or AWS engineering experience. Either one is very helpful. Both are not required.
- Statistics for testing non-deterministic systems: sampling design, sample size, and confidence intervals; precision, recall, F1, and confusion matrices; distributions that are not bell curves; drift detection.
- Generative AI, IDP, and OCR evaluation experience: building ground truth, eval harnesses, versioned prompts, and hallucination detection. Knows the difference between prompt engineering, model tuning, and model training, and how to protect PHI/PII under each.
- Uses AI coding assistants such as Claude Code or Copilot to write test automation and to generate and expand test scenarios.
- Transformer fundamentals at a whiteboard level: tokenization, embeddings, self-attention, next-token prediction, and sampled decoding. Can use that to explain why outputs vary between runs, why long documents get worse results, and why models invent values that look right.
- Automated security testing in a continuous deployment environment.
- Behavior Driven Development (BDD) testing; accessibility testing (WCAG 2.1, Section 508).
- CI tools (GitHub Actions, Jenkins, or similar); Linux scripting.
- Java or C#/.NET for maintaining legacy applications.
- Experience on VA or other federal programs
- Startup experience a plus.
Posted Salary Range
USD $62,000.00 - USD $96,000.00 /Yr.
Company Overview
GovCIO is a team of transformers--people who are passionate about transforming government IT. Every day, we make a positive impact by delivering innovative IT services and solutions that improve how government agencies operate and serve our citizens.
But we can't do it alone. We need great people to help us do great things - for our customers, our culture, and our ability to attract other great people. We are changing the face of government IT and building a workforce that fuels this mission. Are you ready to be a transformer?
What You Can Expect
Interview & Hiring Process
If you are selected to move forward through the process, here’s what you can expect:
- During the Interview Process
- Virtual video interview conducted via video with the hiring manager and/or team
- Camera must be on
- A valid photo ID must be presented during each interview
- During the Hiring Process
- Enhanced Biometrics ID verification screening
- Background check, to include:
- Criminal history (past 7 years)
- Verification of your highest level of education
- Verification of your employment history (past 7 years), based on information provided in your application
Employee Perks
At GovCIO, we consistently hear that meaningful work and a collaborative team environment are two of the top reasons our employees enjoy working here. In addition, our employees have access to a range of perks and benefits to support their personal and professional well-being, beyond the standard company offered health benefits, including:
- Employee Assistance Program (EAP)
- Corporate Discounts
- Learning & Development platform, to include certification preparation content
- Training, Education and Certification Assistance*
- Referral Bonus Program
- Internal Mobility Program
- Pet Insurance
- Flexible Work Environment
*Available to full-time employees
Our employees’ unique talents and contributions are the driving force behind our success in supporting our customers, which ultimately fuels the success of our company. Join us and be a part of a culture that invests in its people and prioritizes continuous enhancement of the employee experience.
We are an Equal Opportunity Employer. All qualified applicants will receive consideration for employment without regard to race, color, religion, sex, gender, gender identity or expression, sexual orientation, national origin, disability, or status as a protected veteran. EOE, including disability/vets.
Posted Pay Range
The posted pay range, if referenced, reflects the range expected for this position at the commencement of employment, however, base pay offered may vary depending on multiple individualized factors, including market location, job-related knowledge, skills, education, experience, and internal equity. The total compensation package for this position may also include other compensation elements, to be discussed during the hiring process. If hired, employee will be in an “at-will position” and the GovCIO reserves the right to modify base salary (as well as any other discretionary payment or compensation program) at any time, including for reasons related to individual performance, GovCIO or individual department/team performance, and market factors.