By Dai Green, Minnesota IT Services (MNIT) Partnering with Minnesota Management and Budget

What was upgraded and why?

Minnesota Management and Budget (MMB) recently completed a successful upgrade of two enterprise systems: 

• Statewide Employee Management Application (SEMA4) – covering payroll, human resources, and benefits.
• Enterprise Learning Management (ELM) systems from Oracle People Tools 8.54 to 8.57. 

This upgrade project also included the application of newer Oracle PeopleSoft Upgrade Manager (PUM) images for each application. 

These applications came into use in 2011 and were beginning to show their age. Also, an upgrade was essential to support desired system enhancements. The refreshed look gives an enhanced user experience for all users. 

MMB and Minnesota IT Services (MNIT) consciously used lessons learned from recent past projects to bring accessibility into the planning and design process from the beginning. The project significantly upgrades complex systems, and relied on: 

• A detailed roadmap. 
• End-user input. 
• A vendor specializing in accessibility.  

Level of effort to bring the project to completion

Upgrading the SEMA4 and ELM applications took 20 months. Project kick off began in January 2020 and we completed a successful upgrade in September 2021. More than 50 people across nine teams from MMB and MNIT participated. There were 10 testing cycles and more than 4000 functional test scripts. In addition, we logged 385 accessibility issues. Each was categorized by severity.  

Working with an accessibility testing service vendor

We partnered with an accessibility vendor to augment our own accessibility Quality Assurance (QA) test staff. The SEMA4/ELM Project followed the state of Minnesota accessibility standard which incorporates WCAG 2.0 AA and Section 508. We completed comprehensive testing on a large scale in a short amount of time. 

The vendor provided a broad perspective from a diverse array of testers. Test methodologies included code examination, and testers who utilize assistive technology and devices. This allowed the MNIT MMB staff to: 

• Review any issues found. 
• Organize the issues into a system that contained a set of predefined criteria for determining the severity of an issue.
• Begin immediate remediation of critical issues. 

Business team leads and the test vendor used an accessibility issue prioritization and criticality metric. This helped MNIT QA to prioritize, document, and manage defects. The business stakeholders and developers reviewed all defects with the QA team, ensured accessibility standards were met, and determined the critical issues. Additional accessibility subject matter experts conducted thorough testing, so MNIT QA staff could focus on identified issues as efficiently and quickly as possible. In addition to vendor testing, the state also evaluated end user experiences by having end users collaborate in the testing process. This key driver ensured accessibility standards were met, and determined the criticality of identified issues that needed to be fixed before the go-live. Usability and accessibility of applications are a prime directive of the state of Minnesota. Working with an accessibility vendor allowed us to rapidly test a large application to meet our project goals and project timeline.

The goal: all users can complete an essential business process 100% of the time. 

Looking forward – inroads to future management

All too often, document and application accessibility is an afterthought in the software development lifecycle. This is also true for the testing of these items. The state of Minnesota takes an active approach to include accessibility in the design process, following usability-by-design concepts.  Staff are committed to the iterative process of producing and maintaining accessible applications and content. 

Maintaining accessible applications needs to be a living process that involves vigilant review of accessibility features, and compatibility of our applications with assistive technology. Projects have definitive start and end dates. This can cause issues to fall through the cracks. Accessible application development requires a commitment to ongoing testing from release management and upgrades, to technology evolution. 

Comments from our leadership 

“Reviewing systems for accessibility is a process that continues even after a project is complete. Accessibility should be examined when updates to technology and business enhancements change the end user experience.… Accessible systems improve the end user experience for all.” 

- Laurie Hansen, MMB Finance Services Director

“Accessibility is very important to us at MMB and at MNIT. This is why we’ve made a concerted effort to enhance and grow our capabilities to test and validate our statewide systems’ ability to be used effectively by everyone.” 

- G. Bruce Yurich, MNIT Chief Business Technology Officer