Office of Inspector General
City of Chicago
Report of the Office of Inspector General: *************************
DEPARTMENT OF FAMILY AND SUPPORT SERVICES HOMELESS SER VICES FOLLOW-UP INQUIRY
1 M
o l—o
ID—I
o
'-o •¦
October 2016
866-IG- TIP LINE (866-448-4 754) www.cliicacoinspeclorsenernl.oro
Joseph M. Ferguson Inspector General
OFFICE OF INSPECTOR GENERAL City of Chicago
740 N. Sedgwick Street, Suite 200 Chicago, Illinois 60654 Telephone: (773)478-7799 Fax: (773)478-3949
October 24, 2016
To the Mayor, Members of the City Council, City Clerk, City Treasurer, and residents of the City of Chicago:
The City of Chicago Office of Inspector General (OIG) has completed a follow-up to its August 2015 audit of the Department of Family and Support Services' (DFSS) homeless services. Based on the Department's responses, OIG concludes that DFSS has begun implementation of corrective actions related to the audit findings.
The purpose of the 2015 audit was to determine how DFSS selected and monitored homeless service providers, called "delegate agencies." Our audit found that, due to multiple errors in DFSS's evaluation of Request for Proposal (RFP) respondents, DFSS may not have selected the most qualified agencies. In addition, because of gaps in the audit process, DFSS failed to hold agencies fully accountable for program inadequacies and inaccurate reporting.
Based upon the results of our audit, we recommended that DFSS should,
improve reviewer training and evaluate its RFP Scoring Tool to determine where it could minimize the probability of human error;
improve its quality control procedures;
design its delegate agency audit tool to ensure that all known problems are identified and categorized appropriately and consistently;
more thoroughly examine the scope and causes of inaccurate performance data; and
hold delegate agencies fully accountable for misreporting performance data.
In its response to the audit, DFSS described a number of corrective actions it would take.
In September 2016, OIG inquired with DFSS regarding the status of the corrective actions the Department committed to and any other actions it may have taken. On the following pages we have summarized the three original audit findings and recommendations, as well as the Department's response to our follow-up inquiry.
Based on DFSS's follow-up response, OIG concludes that DFSS has implemented the corrective actions to address one finding concerning the Department's audit tool, and has begun implementing the corrective actions to address the remaining two findings regarding RFP scoring and delegate agency performance reporting. Once fully implemented, OIG believes the corrective actions reported by DFSS may reasonably be expected to resolve the core findings
Website: www.cliicnuoinspectorijenernl.oru
noted in the original audit. Based on the agency's responses, OIG expects this will be accomplished in 2017.
We thank the staff and leadership of DFSS for their cooperation during the original audit and responsiveness to our follow-up inquiries.
Respectfully,
Joseph M. Ferguson Inspector General City of Chicago
Website: www.chicauoinspectorgeneral.org
OIG File #16-0378
DFSS Homeless Services Audit Follow-Up Inquiry
FolIow-Up Results
In September 2016, OIG followed up on an August 2015 audit of DFSS's homeless services.' DFSS responded by describing the corrective actions it has taken since receiving the original audit and provided supporting documentation. We summarize the three original findings, the associated recommendations, and the status of the Department's corrective actions below. OIG's follow-up inquiry did not observe or test implementation of the new procedures, and thus makes no determination as to their effectiveness, which would require a new audit with full testing of the procedures.
OIG uses four categories for Status of Corrective Action:
Implemented - The department has implemented actions that may reasonably be expected to resolve the core findings/concerns noted in the original audit.
Partially Implemented - The department has implemented actions in response to the audit, but the actions do not fully address the findings/concerns raised in the original report.
Pending Implementation - The department has initiated action plans that, if fully implemented, may reasonably be expected to resolve the core findings of the original audit. However, the department has not completed implementation.
Not Implemented - The department has not initiated or implemented any actions responsive to OIG's findings.
Finding 1: DFSS scored delegate agency applications inaccurately and did
not follow its own guidelines for resolving inconsistent scores.
OIG Recommendation: We recommended that DFSS improve reviewer training and
evaluate its Scoring Tool to determine where it can minimize the probability of human error.
In addition, we recommended that DFSS improve its quality control procedures, noting that potential improvements could include documenting clear procedures for resolving scoring differences that exceed a specific threshold, and reallocating resources as needed to ensure resolution.
Status of Corrective Action: Pending Implementation. Since the time period examined in the
original audit, DFSS adopted the City's Cyber Grants software to manage delegate agency applications. The software automatically calculates points awarded by application reviewers, eliminating the chance of human error in calculating application scores.
1 The 2015 audit report is available on the OIG website: > content/tiploads/2015/08/DFSS-Homcless-Serviccs-Audit.pdf.
Page 1 of 3
OIG File 1116-0378
DFSS Homeless Services Audit Follow-Up Inquiry
DFSS has also begun to stagger the Community Development Grant Application (CDGA) RFP release schedule for its various programs in order to reduce the volume of applications requiring simultaneous review. During 2016, DFSS released CDGA RFPs for its Youth, Workforce, Human, and Senior Services programs, and the Department plans to release the next Homeless Services CDGA RFP in spring 2017. For the CDGA applications received to date in 2016, DFSS designated a staff person to flag any instances where the application scores assigned by two reviewers differed by 20 or more points. In such cases, a third reviewer was assigned to read the application.
DFSS plans to use a webinar to train the 2017 Homeless Services CGDA application reviewers on the scoring tool, software, and program details.
FINDING 2: DFSS allowed known audit findings to go unrecorded because
of a flaw in the design of DFSS's audit tool.
OIG Recommendation: We recommended that DFSS design its audit tool to ensure that all
known problems are identified and categorized appropriately and consistently.
Status of Corrective Action: Implemented. DFSS stated that it "revised protocols and provided
training to Program Auditors on how to classify Findings outside of an audit sample. These changes allow DFSS to ensure Program Auditors accurately and consistently categorize problems identified during program monitoring visits." DFSS provided to OIG copies of the Program Monitoring Guidance and Program Review Monitoring Instrument distributed to all Program Auditors. It also provided five examples of program audits conducted between 2015 and 2016 wherein the revised audit tool prompted Program Auditors to record findings that they likely would not have recorded using the old audit tool.
Finding 3: DFSS required delegate agencies to report on their
performance but did not hold them fully accountable for inaccurate reporting.
OIG Recommendation: We recommended that DFSS more thoroughly examine the scope
and causes of inaccurate performance data. For example, when DFSS finds inaccurate data for one quarter, it should consider examining additional quarters as well, and determining whether data was intentionally or unintentionally misreported. We also recommended that DFSS hold delegate agencies fully accountable
Page 2 of 3
OIG File #16-0378
DFSS Homeless Services Audit Follow-Up Inquiry
for misreporting performance data. One way of achieving this could be to elevate misreporting from an audit Concern to a Finding, which would require agencies to take corrective action to prevent future misreporting. Elevating misreporting to a Finding would also ensure that past misreporting is reflected on the Scoring Tool and thus taken into consideration during agency selection.
Status of Corrective Action: Pending Implementation. DFSS endeavored to compare delegate
agencies' quarterly performance reports to data reported in the Homeless Management Information System (HMIS), as it had intended to do in response to OIG's audit. However, the Department found that the HMIS data was not strictly comparable and did not capture all the necessary information. HMIS is a U.S. Department of Housing and Urban Development-mandated tool that is shared with other agencies, and DFSS cannot unilaterally change HMIS to suit its needs.
Therefore, DFSS plans to take a different approach to delegate agency performance accountability in 2017. The Department will train delegate agencies via webinar on the requirements for accurate documentation and performance reporting. The Program Monitoring Unit will conduct monitoring visits beginning in the second quarter of 2017, and will pilot a modification to the monitoring tool that will categorize misreporting as a Finding rather than a Concern. If reporting mistakes are discovered, DFSS will conduct additional trainings to prevent their recurrence, and conduct follow-up site visits to determine if mistakes persist.
More broadly, DFSS is implementing a new Strategic Framework that "will transition DFSS to a more outcome-based model that focuses on how many people leave belter off after receiving services from DFSS as opposed to tracking outputs. The Strategic Framework consists of a refreshed mission, priorities and goals, along with a plan for how DFSS will measure, report on, and review the goals in the years to come; use them to make decisions' and drive greater collaboration within DFSS." The Framework includes measurable goals for improving delegate agencies' performance over time.
Page 3 of 3
City of Chicago Office of Inspector General
Public Inquiries Rachel Levcn (773) 478-0534 rleven@chicagoinspectorKeneral.org
To Suggest Ways to Improve City Government Visit our website:
improve-city-government/
To Report Fraud, Waste, and Abuse in City Programs Call OIG's toll-free hotline 866-IG-TIPLINE (866-448-4754). Talk to an investigator from 8:30 a.m. to 5:00 p.m. Monday-Friday. Or visit our website:
waste-fraud-and-abuse/
Mission
The City of Chicago Office of Inspector General (OIG) is an independent, nonpartisan oversight agency whose mission is to promote economy, efficiency, effectiveness, and integrity in the administration of programs and operations of City government. OIG achieves this mission through,
administrative and criminal investigations; audits of City programs and operations; and reviews of City programs, operations, and policies.
From these activities, OIG issues reports of findings and disciplinary and other recommendations to assure that City officials, employees, and vendors are held accountable for the provision of efficient, cost-effective government operations and further to prevent, detect, identify, expose, and eliminate waste, inefficiency, misconduct, fraud, corruption, and abuse of public authority and resources.
Authority
The authority to produce reports and recommendations on ways to improve City operations is established in the City of Chicago Municipal Code § 2-56-030(c), which confers upon the Inspector General the following power and duty:
To promote economy, efficiency, effectiveness and integrity in the administration of the programs and operations of the city government by reviewing programs, identifying any inefficiencies, waste and potential for misconduct therein, and recommending to the mayor and the city council policies and methods for the elimination of inefficiencies and waste, and the prevention of misconduct.