Appendix C

Education Data for the District of Columbia

The data available for use in an evaluation of District of Columbia (DC) Public Schools include both that collected by the city itself (by various offices, including DC Public Schools [DCPS], the Office of the State Superintendent of Education [OSSE], and the Office of the Chief Financial Officer [OCFO]), and data collected by National Center for Education Statistics (NCES) of the U.S. Department of Education.

DISTRICT OF COLUMBIA DATA SOURCES

The District of Columbia has a number of data systems related to schools. DCPS provided the committee with a list of current databases housed within DCPS, OSSE, and the OCFO, and a list of evaluations or studies currently under way or recently completed (see tables below, as of March 2011). The DCPS systems include all of the basic student attendance, achievement, attainment, and tracking systems (e.g., DC STARS, ThinkLink Online), reading interventions (e.g., Read 180), human capital management systems (e.g., IMPACT), and management, operations, and finances systems (e.g., Transportation Management System). OSSE’s sources of data include the State Longitudinal Education Data (SLED) warehouse (not yet operational) and a tracking system for individualized education plans (IEPs) required under IDEA. The OCFO data systems include the procurement and accounting systems. The evaluations include on-time studies, such as the City Year Evaluation, as well as ongoing assessments, such as the stakeholder surveys for which DC reports data every

year at the school and district levels (see http://dc.gov/DCPS/About+DCPS/Satisfaction+Stakeholder+Surveys [accessed March 2011]).

We were not able to review all of these data systems, but have a few comments. A report recently released by the Council of the Great City Schools (2010),¹ for example, listed as its first finding about DC that there were “significant challenges to data quality” and “there was a lack of universal practice and oversight by the district in creating data comparable across DCPS schools and ensuring accurate information within the system. For example, there was no central control over student ID creation and no validations (automatic or hand-checked) to the system to guard against duplication.”

The DC data and accountability chief made a presentation to the Committee, in which she acknowledged that, although the office had made significant progress in improving data collection efforts, much more needs to be done. She cited as an example of problems she found on taking office the formerly standard practice in DC’s student tracking system of counting students as present unless otherwise noted by the school, which led to greatly overstated attendance rates.

Quality issues have also been raised with other DCPS databases. For example, in 2007, independent monitors of DC’s special education system said of the special education data that “Most [case analyses] require tracking down the student at a school that differs from the one listed as the attending school in [the data system]…[the system] does not meet standard system requirements of…data quality control[.]… There are several hundred ‘lost students.’…No one is really sure where they are at any one time.” In 2009, the District also terminated its contractor on the building of their State Longitudinal Data System for default.

These preliminary findings do not in any way suggest that all of the district data are of poor quality or unsuitable for use in a thorough evaluation. They do suggest that, as would be done at the beginning of any research study, the evaluation begin with careful consideration of the quality of the data available to support investigation of the specific research questions and methods envisioned.

Table C-1, below, is the list of data sources related to education that DCPS staff provided to the committee. These include data collected by each of the relevant city agencies.

DCPS also provided information about data being collected by the National Center for the Analysis of Longitudinal Data in Education

_______________

¹Smerdon, B., and Evan, A. (2010). The Senior Urban Education Research Series, Volume 1: Lessons for Establishing a Foundation for Data Use in DC Public Schools. Washington, DC: Council of the Great City Schools. See http://www.cgcs.org/publications/DC_FellowReport2010.pdf [accessed March 2011].

Research (CALDER), through a memorandum of understanding with DC and supported by the U.S. Department of Education. This project links data from the multiyear enrollment automated database (MEAD), assessment data, student residential information, and school files. Tables C-2 and C-3 list the sorts of information being collected through this project.

DATA FROM THE NATIONAL CENTER FOR EDUCATION STATISTICS

The federal data collections within NCES of the U.S. Department of Education that include information about DC’s schools and students are the Common Core of Data (CCD), the Schools and Staffing Survey (SASS), the National Assessment of Educational Progress (NAEP), and the Trial Urban District Assessment (TUDA). Below is a summary of the measures included in each.

Common Core of Data (CCD)

The CCD survey annually collects data about all public elementary and secondary schools, all local education agencies, and all state education agencies throughout the United States. CCD contains three categories of information: general descriptive information on schools and school districts; data on students and staff; and fiscal data. The general descriptive information includes name, address, phone number, and type of locale; the data on students and staff include selected demographic characteristics; and the fiscal data cover revenues and current expenditures. Most of the data are compiled by state education agencies and sent to the Department of Education. The CCD data are comparable across all states. Data are also collected for DCPS, and, since 2004, for charter schools operating in DC.

Specific data include the number of students by grade level; fulltime staff by major employment category; high school graduates and completers in the previous year; average daily attendance; school district revenues by source (local, state, federal); and expenditures by function (instruction, support services, and noninstruction) and subfunction (school administration, etc).

Then Acting Commissioner of NCES, Stuart Kerachsky, made a presentation to the committee about these data. He noted that collecting data for DC is more challenging than doing so for other jurisdictions for several reasons. Both the quality and the timeliness of the DC data have not been comparable to those of other states. For example, the District’s data exhibit a high percentage of missing data, especially for staff categories. In addition, the average tenure of the fiscal data coordinator in DC is markedly shorter than the average for states, which hampers continuity. Finally, new

TABLE C-1 List of Education-Related Data Systems Provided by DCPS

		DCPS Data Systems Inventory
#	Title	Owner (Agency)	Description
1	Genesis Earth	DCPS	Head Start workflow management
2	Work Sampling Online	DCPS	Head Start comprehensive child assessment tool
3	DCPS Administrator	DCPS	Used to process principal and assistant principal applications
4	Gov Delivery	OCTO	Used to send bulk emails to parents, etc.
5	Imagine Learning	DCPS	Reading intervention program for certain English language learner 3rd through 5th grade students. Used as learning aid directly by students
6	Early Steps and Stages Tracker	OSSE	Early Stages IDEA Part C tracking. Tracks students identified as possibly having special needs (birth - 2yr)
7	Early Stages Tracking, Monitoring, and Reporting	DCPS	Early Stages IDEA Part B tracking. Tracks students identified as possibly having special needs (2yr - 4yr 9mo)
8	DC STARS	DCPS	Student Information System (SIS)
9	Blackman-Jones Database	DCPS	Tracks Blackman-Jones statistics for consent decree reporting and case management
10	CAASS	DCPS	Student access control system. Tracks students as they enter school buildings, currently used for school security
11	DCPS Public Website	DCPS	Public website for DCPS
12	SEDS (Easy IEP)	OSSE	Individualized Education Program (IEP) management system
13	FileNet	OCTO	Document imaging system
14	PD Planner	DCPS	Online catalog and activity management system for professional development offerings for DCPS educators and employees
15	PASS	OCTO	Procurement management system
16	SOAR	OCFO	System of accounting and reporting (financial mgmt system) - General ledger, accounts payable, budget, fixed assets, accounts receivable, cash management, inventory management

SOURCE: Personal communication from DCPS, March 2011.

TABLE C-2 Information on DC Schools Being Organized by CALDER

SOURCE: Jane Hannaway presentation to committee, prepared by the National Center for Analysis of Longitudinal Data in Education Research (CALDER).

TABLE C-3 Links That Can Be Established Using the DC Data, By Year

SOURCE: Jane Hannaway presentation to committee, prepared by the National Center for Analysis of Longitudinal Data in Education Research (CALDER).

strategies have been needed each year to make the data comparable to those for other states and prior years, for example, to account for changing classifications of charter schools.

Schools and Staffing Survey (SASS)

The SASS is an integrated sample survey comprised of four questionnaires targeting public and private schools, school districts, and principals and teachers. Data collection at the state level began in 1987-1988 and again in 1990-1991, 1993-1994, 1999-2000, 2003-2004, and 2007-2008, resulting in public release and restricted-use datasets. The next SASS collection is scheduled for the 2011-2012 and 2015-2016 school years.

One year following each administration of the SASS survey, a follow-up questionnaire is administered to the initial group of teachers to determine the numbers of those who have left their positions or have moved on to other positions. For the first time in 2007-2008, this follow-up questionnaire, known as the Teacher Followup Survey (TFS), was also used to monitor the movement of first-year teachers. In addition, the year after SASS 2007-2008, a state-level Principal Follow-up Survey (PFS) was conducted on all principals interviewed in SASS.

The SASS 2007-2008 collection includes 104 DC traditional public schools, their principals and school libraries at the state level, as well as an additional sample of public charter schools from 16 districts (many DC charter schools are their own districts). Finally, a sample of three to eight teachers per school based on school enrollment is also included.

The SASS provides data on

• Districts, including enrollment, teachers, principals, count of newly-hired teachers, teacher schedules and salaries, types of benefits offered to teachers, number of newly-hired principals, principal salary schedule, number of contract days for teachers and principals per year, existence of a collective bargaining agreement or other type of agreement, school choice policies, high school graduation requirements, presence of incentives to recruit or retain teachers, poor performance dismissals or contract nonrenewals, high school graduation policy;
• Schools, including staffing counts, Title I teacher count, grade levels offered, student enrollment by race/ethnicity, IEP and LEP students, migrant students, school year length, programs/services offered, public school designation as charter or traditional;
• Library Media Centers, including services, policies, expenditures in previous year, types of holdings and equipment, assistive technology availability;

• Principals, including demographics, salary, hours worked per week, number of instructional hours students receive per week at grades 3 and 8 for core academic subjects, attitudes and school climate, policies on school safety; and
• Teachers, including demographics, salary, workload, preparation, certification, teaching assignment, grades taught, number of students taught (average class size can be calculated), professional development, attitudes on school climate.

National Assessment of Student Progress and Trial Urban District Assessment

The National Assessment of Educational Progress (NAEP) collects academic achievement data and related background information. Table C-4 shows the DC data available from NAEP.

The Trial Urban District Assessment (TUDA) is designed to explore the feasibility of using NAEP to report on the performance of public school students at the district level, for those districts selected to be a part of TUDA (see Table C-5). As authorized by federal law, NAEP has administered the mathematics, reading, science, and writing assessments to samples of students in selected urban districts public schools.

TABLE C-4 Available NAEP Data

Subject	National Only Results	National and State Results	Participating Urban District Results
Arts		—	—
Civics		—	—
Economics		—	—
Geography		—	—
Long-Term Trend		—	—
Mathematics	—
Reading	—
Science	—
U.S. History		—	—
Writing	—

NOTE: Unshaded rows are areas for which DC data are available.
SOURCE: Nation’s Report Card, available: http://nationsreportcard.gov/about.asp [accessed December 2010].

TABLE C-5 District Assessment Participation

	2002 Reading and Writing	2003 Reading and Mathematics	2005 Reading, Science, and Mathematics	2007 Reading, Mathematics, and Writing	2009 Reading, Mathematics, and Science
Atlanta Public Schools
Austin Independent School District
Baltimore City Public Schools
Boston School District
Charlotte-Mecklenburg Schools
Chicago Public Schools
Cleveland Metropolitan School District
Detroit Public Schools
District of Columbia Public Schools
Fresno Unified School District
Houston Independent School District
Jefferson County Public Schools (KY)
Los Angeles Unified SchoolDistrict
Miami-Dade County Public Schools
Milwaukee Public Schools
New York City Public Schools
San Diego Unified SchoolDistrict
School District of Philadelphia

NOTES: Beginning in 2009, if the results for charter schools are not included in the school district’s Adequate Yearly Progress (AYP) report to the U.S. Department of Education under the Elementary and Secondary Education Act, they are excluded from that district’s TUDA results. See more information on the comparability of the 2009 NAEP design. Due to an insufficient sample size, the District of Columbia did not participate in the science assessment in 2005 and 2009 and the writing assessment in 2007.
SOURCES: U.S. Department of Education, Institute of Education Sciences, National Center for Education Statistics, National Assessment of Educational Progress (NAEP), various years, 2002-2009 assessments. Nation’s Report Card, TUDA, available: http://nationsreportcard.gov/tuda.asp [accessed December 2010].

NAEP and TUDA provide scale scores and achievement level data, along with background information and allow for trend analyses within states and districts, and comparisons with others. NAEP and TUDA also survey school administrators regarding information about the school and teachers regarding their educational background, experiences, and instructional practices. Every 4 years a high school transcript study is conducted.

In 2009, charter school results were included in the state-level NAEP assessment but were not included in the district-level TUDA results for DC. In that year, the math sample for DC 4th grade included approximately 1,900 students in NAEP and 1,400 in TUDA; for 8th grade approximately 1,800 students in NAEP and 900 in TUDA. State-level (NAEP) and district-level (TUDA) math and reading assessments will be next administered for DC in 2011, 2013, and 2015 (schedule subject to change, see table). The schedule for NAEP and TUDA administrations is shown in Table C-6.

TABLE C-6 Schedule of NAEP and TUDA Assessments, 2005-2017

Last Updated March 10, 2010.
^aUpdated or new framework is planned for implementation in this subject. In the case of subjects for which frameworks are already adopted, the Board will decide whether a new or updated framework is needed for this assessment year.
^bNew framework for grade 12 mathematics only, in 2009.
^cFor 2009, there is a pilot study of state-level results, for which 11 states volunteered.
NOTES: At the national level, grades tested are 4, 8, and 12 unless otherwise indicated, except that long-term trend assessments sample students at ages 9, 13, and 17. The Governing Board intends to conduct assessments at grade 12 in world history and foreign language during the assessment period 2018-2011.
SOURCE: See http://nces.ed.gov/nationsreportcard/about/assessmentsched.asp [accessed March 2010].

This page intentionally left blank.