USIS thought it just needed an extra hand for a few weeks. In
the end, it got much more than that from StoneHenge Partners.
Headquartered in Falls Church, VA, USIS is an information and
security services company serving human resources, insurance, and
government agencies. It is the largest supplier of background
investigations to the federal government, and the preeminent source
of criminal background checks and drug screens to businesses
nationwide.
The problem
A mission-critical application for USIS is its Criminal Records
data warehouse, known internally as Bulk Crim, by which it performs
criminal-record background checks on thousands of individuals each
day on behalf of 300+ clients. Parts of the legacy system were more
than 20 years old, running on a VAX/VMS mainframe so obsolete that
parts were being scavenged from junked systems to keep it
running.
Gathering criminal records is a messy, hodgepodge process
involving more than 250 sources -- state, district and county
courts. Some sources post records on websites while others offer
flat files; some sources update records daily while others only
once a month; some sources serve up thousands of tiny files while
others drop a single multi-gigabyte data dump.
The solution
USIS decided to build a new application for Bulk Crim, using the
latest Microsoft technologies: C#, .NET, and SQL Server Integrated
Services (SSIS) package on SQL Server 2005. It also defined two
tools to collect the information: parsers to digest flat files, and
web scrapers to collect online records. In 2007, business
requirements were defined, and a consulting firm in India won the
bid to build the new app.
In mid-2008, the India firm began delivering custom parsers and
web scrapers for the app. Under terms of the contract, USIS had one
week to test and approve/reject each parser and scraper. To
help with the overflow, they brought in one StoneHenge consultant,
then another. At the peak of development, five StoneHenge
consultants were engaged in the project.
The result
As the project progressed, the role of StoneHenge consultants
grew from purely tactical day-to-day work into strategic
consulting, partly because the development team encountered
problems that were out of scope of the India contract. For
example:
- The state of Oklahoma blocked the USIS web scraper because it
was so efficient that it was perceived to be a web service attack.
The team analyzed the problem, and came up with a solution to bring
the scraper within the state's parameters.
- The USIS Quality Assurance department had trouble understanding
how all of the piece fit together, so a StoneHenge QA expert
consulted with them to make their testing more effective.
- The team discovered that some data files were so large that
they risked choking the system. StoneHenge consultants analyzed the
problem and developed a solution to manage the data flow and
minimize the risk.
- The initial design called for all business logic to be handled
in SQL Server stored procedures. However, because of the large
amounts of data, that posed a risk of crashing SQL Server.
StoneHenge consultants analyzed the problem and developed a Windows
service (QA Processor) that would pull data in more efficient
manner.
- Even though the system was intended to be automated, human
intervention was required to handle exceptions and errors. The team
developed a service, called QA Processor, and 12-screen user
interface that allowed business users to monitor data quality,
handle exceptions, and correct errors.
"We did a demo for the business users, so we could gather
their feedback and create an interface that worked for them," said
Jason Hammock, StoneHenge Partners technical lead. "We didn't have
to do it - it wasn't part of the "job" - but when I see a need, I
want to solve it. You can't just throw code over the wall."
The project was considered a success. This has positioned USIS
in a better position to integrate all of their tools, and has built
a solid foundation for their core business system. StoneHenge
Partners continues to consult with USIS on a variety of
projects.