Saturday, March 5, 2011

Statistics and more context

Chairperson Stevens presented his proposed LCPS budget to the Board of Supervisors on March 2, which included a number of statistics designed to win approval of the plan. These are questions I think should be asked about some of those statistics.

"The Center for American Progress rated districts of all sizes nationally using data from the 2007-2008 school year, and rated LCPS among the top-performing school districts for efficiency & results. I'll have more details in a future post."

We do need those details. When I read the report in January I was struck most by this statement:
"Our nation’s school system has for too long failed to ensure that education funding consistently promotes strong student achievement. After adjusting for inflation, education spending per student has nearly tripled over the past four decades. But while some states and districts have spent their additional dollars wisely—and thus shown significant increases in student outcomes—overall student achievement has largely remained flat. And besides Luxembourg, the United States spends more per student than any of the 65 countries that participated in a recent international reading assessment, and while Estonia and Poland scored at the same level as the United States on the exam, the United States spent roughly $60,000 more to educate each student to age 15 than either nation."
Yes, LCPS appears to do better at providing a return on investment than many schools in Virginia, but it is by no means a leader; it is pretty much in the middle of the pack. For instance, locally, the report lists Fauquier, Clarke, Prince William, and Stafford counties as providing a significantly stronger education ROI to its taxpayers. More importantly, it's hard to miss the Center's overall condemnation of public schools' efforts at efficiency and ROI. Loudoun is in the middle of a not-so-well rated pack.

Bear with me a minute, and take the time to read the report's thesis, something I think we should keep in mind when citing the report--
"When successful businesses want to improve performance and boost efficiencies, they focus on creating the conditions for organizational change. They use data to identify problem areas, create short and long-term goals, and engage their employees to sustain transformations and nurture further innovation. Such approaches have long worked for the private sector, and there’s clear evidence that the techniques can help drive better performance in large, public organizations as well.

"But schools and districts have long been effective at deflecting or watering down meaningful change in order to protect entrenched bureaucracies and interests. And even reform-minded school administrators often confuse merely novel techniques with successful ones and dash from one educational fad to the next without tracking their efficacy. To increase productivity, school leaders will need to fundamentally reinvent the way that they do business and create an outcomes-based school culture that sets high goals—and gives employees the strategies to achieve them.

"That will entail doing away with obsolete traditions and ineffective programs, to be sure. But it will also require schools and districts to embrace transformational ways of delivering a cost-effective education that reduces spending while boosting performance. The goal must be nothing short of a breakthrough in performance that guarantees that every dollar produces high achievement for all students."


"LCPS has become more efficient, reducing the ratio of non-school-based positions from 10.0 per 1,000 students to 9.2 per 1,000 students. Our proposed FY12 budget, has a higher share of school-based employees than any other NoVA district."

My concern with this statement is two-fold: (1) A simple math calculation seems to show that this change could primarily be the result of not hiring more staff during the past couple years as student population grew; and (2) while that might result in greater efficiencies, this is an awfully narrow definition of efficiency, and just saying that we did more with less doesn't mean our processes became more efficient.

In fact, it could be argued it was the voters who limited the past few budgets who achieved this "efficiency," not just LCPS. But again, this doesn't prove LCPS is more efficient; it could be that it is only as efficient as it was two years ago, but being so with less staff. This statement begs more support than just a simple math calculation. I'm definitely interested in hearing more on this.

"School lunch subsidies have dropped from $900,000 to $0. Lunch prices have climbed from $2.10 to $3.00"

My first thought may seem a non sequitur, that the $900,000 is close to the amount LCPS is not going to collect in AP Exam fees next year (an amount that had to have been in the first budget drafts). Factor in that we will likely have to refund the $900k from this year, and possibly the $900k from last year, that lunch subsidy savings doesn't look all that significant. I'm almost afraid to ask how seriously the AP and Honors courses grade weighting committee considered the legality of these fees a couple years ago?

This raises another question about what we mean by efficiency: How is shifting the revenue collection method from one path to another necessarily an indicator of cost savings and process improvement? Chairperson Stevens may be saying that the total costs of school lunches decreased, but the juxtaposition of the subsidy elimination with a statement that families pay more for each lunch can lead us to believe that there is a relationship between the two statements. Loudoun families could still be paying the same amount for lunches, so where's the savings, where's the efficiency? Whether the funds come out of our property taxes or our wallets each morning makes no difference here. I could be wrong, so let's get more context.

"Average class sizes have increased."

Please show me the scientific research that supports the idea that class size has a significant impact on classroom learning when compared to other factors also within the control of LCPS (other than NEA assertions). Period.

"The average Loudoun tax bill has dropped from $5,307 to $5,244 at the proposed tax rate of $1.32"

Another simple math calculation shows this drop is barely %1. I'm concerned that property owners who have seen their homes devalued in the double-digit percent amounts aren't going to buy this as a reason to support the proposed budget.

Statistics are important, but so is context. Let's get some more.

19 comments:

  1. More context from that same report:

    Our emphasis on educational productivity does not mean that we believe that
    lawmakers should spend less on education. Quite the opposite. Transforming our
    schools will demand both real resources and real reform, and our project is an
    argument for dramatically improving our nation’s school system so that dollars
    create results.

    Also, Although Loudoun appears to be "middle of the pack" in per pupil spending, it seems to rate higher in achievement.

    ReplyDelete
  2. Yes, but we must ask if spending (or not spending) drives that achievement.

    I have no problem with spending more dollars on education, and agree with the report on that. But I also agree with the report that the investment must improve efficacy and outcomes as well). But that means we must "use data to identify problem areas, create short and long-term goals ... [and] engage employees to sustain transformations and nurture further innovation." We aren't doing that.

    We need to ask whether the "achievements" LCPS touts are a result of its efforts or primarily driven by the affluence of the families of Loudoun.

    I advocate nothing short that "reinvention:"

    "To increase productivity, school leaders will need to fundamentally reinvent the way that they do business and create an outcomes-based school culture that sets high goals—and gives employees the strategies to achieve them."

    ReplyDelete
  3. (But that means we must "use data to identify problem areas, create short and long-term goals ... [and] engage employees to sustain transformations and nurture further innovation." We aren't doing that.)

    And you know LCPS is not doing this? How? Do you mean every school? Your school? What is your role/authority to make that judgement on an entire school system? So the quality of teachers in LC doesn't really matter? All the work that schools put into analyzing their data when developing their SIP plans...no point?

    ReplyDelete
  4. Anonymous at 3:33, please keep asking the questions, but not just of me, but of LCPS as well.

    As for me, I have participated in a number of district-wide and school programs in which substantial investments in time, resources, and dollars have been made (and still are), but for which no measurements of stated goals are made. One program, focused on assisting targeted student populations be successful in college, has never checked on how the thousands of students who have "graduated" from the program have done in college. The program spends hundreds of thousands of dollars every year on its efforts, but has no process in place (or apparent interest) to measure whether those efforts pay off.

    We should also ask whether the numerous anti-bullying programs in place are achieving results. Yes, this is a problematic area for which to identify results, but do any of these programs even have a process in place for measuring progress toward outcome-based goals? For example, at one such school-based effort in which tens of thousands of dollars are spent annually, the director of guidance was asked if it might be a good idea to see if the incidence of fights, acts against campus, detentions, and suspensions had changed during the lifetime of the program. His response? "Why would we want to do that?"

    There may be some programs at LCPS that genuinely measure authentic outcomes. Feel free to inquire of your own school's anti-bullying program, for instance. Ask the staff what measurable goals are set and how often are they assessed. Ask if the leadership routinely reviews and adjusts the program to meet those goals. If you receive satisfactory responses, then let me know; such a model needs to be praised and advertised.

    One other program in which we're all involved was the focus of a Washington Post article a week or so ago. It's a massive program costing county taxpayers hundreds of millions of dollars every year. Our employment.

    I'm hoping that the Assistant Superintendent of Personnel Services was misquoted in that article, otherwise we really must wonder whether LCPS really measures any "goals." Consider that teacher attrition and retention are incredibly important issues in public education, primarily because our attrition rates are so much higher than those of most other businesses. Hiring, training, and replacement of school employees costs schools millions of dollars.

    You would think that quite a bit of focus on why teachers leave would be a given. Afterall, every other employer I've known of has asked resigning employees their reasons for departure for decades.

    So it comes as quite a surprise when Mary Ann Hardebeck states that we only started asking that question last year. I would hope that a key component of any employee retention program would be to understand why employees stay and leave; and we only start asking them why last year?

    As I said, keep asking those questions. I don't have all the answers, but being asked those questions often helps me get closer to them. More importantly, however, make sure the people who work for us have the answers -- or at least are looking for them.

    ReplyDelete
  5. Erv Addison: you are awesome. Why don't you run for the BOS or school board? You might be surprised at how many voters would gravitate to your reasonable view on the LCPS budget vs John Steven's "tax increases every year forever to fund a bloated, wasteful, over-paid LCPS bureaucracy"

    ReplyDelete
  6. John Stevens's views and my views actually differ very little. What's different between us are the roles and methods we've chosen to embrace. Keep in mind I have taken on the mantle of a nom de plume, which allows me to advocate and work for my goals simultaneously in two different environments and voices. Rest assured that my views and goals are identical in both roles, it's just that I consciously take on a more aggressive position in public as Erv and a more collegial persona at my high school (I hope). Evidence that both personae are consistent occurred the other day when a colleague, who was reading one of my letters in The Loudoun Times, suddenly stopped and exclaimed to me, "You're Erv Addison!"

    Chairperson Stevens, on the other hand, doesn't have that luxury. His personal and very public lives are both very visible. Consequently, his strategies for achieving education success in Loudoun will be different — as they often are for most public officials. He must balance his talents, skills, beliefs, and goals with those of the other eight members of the School Board, as well as those of the nine Supervisors — and Mr. Hatrick and his administration and the 6,000 employees he helps leads, and the 60,000 students whose futures he impacts, as well as the 310,000 residents of Loudoun County who pay for and consume his services.

    While I have run for office and worked on political committees (and will continue to do so down the road), change and community achievement depend on more than being a office holder. They require complementary roles and efforts. At this time I choose to work within the culture I am trying to change, through leading by example, taking on both leadership and "gofer" roles in school and district programs, teams, and committees, establishing and extending dialogues with colleagues, friends, and not-so-much friends which I hope will change the way they think and act. Do I think my roles are easier than the Chairperson of the School Board's? Not necessarily: I think there is much about public education that needs to be changed, and Now! not tomorrow; so you can imagine how frustrating my average workday must be at times as I experience the often inimical learning environment we inflict on children. You can imagine as well how many co-workers avoid conversations with me, knowing how much my (very tactfully expressed) views either different from theirs or, at least, would require them to act differently.

    And, of course, these interactions change me as well — which is necessary and desirable. Do I always agree with Chairperson Stevens? No. And that is a requisite for change, rubbing brains together to transform the system. I do, however, think we see eye-to-eye on more things than not, and that for real change to happen, our many roles and perspectives are needed.

    Thank you for thinking my ideas and positions are awesome; that will make going into work tomorrow that much easier. Please, however, poke holes in my ideas as you do those of others playing different roles.

    ReplyDelete
  7. A problem I have with your post, Erv is that you are pulling context from The Center for American Progress report that is referring to education in our nation. People reading your post may think that LC is "failing to ensure that education funding consistently promotes strong student achievement." Obviously some people believe this, but the report is NOT singling out LC. On the contrary...the same report is rating LCPS amoung the top-performing school districts.

    As you stated...it costs alot to recruit, hire and train new teachers. Something that needs to be taken into consideration as we continue to have to build new schools and hire more teachers...we need to keep the teachers we have.

    You seem to believe LC success is largely due to it's affluence (despite it's success in less affluent areas). I teach in an affluent area and we make AYP with no problem, but it doesn't stop us from looking at our subgroup scores (data) and planning strategies, interventions and programs to close any gap between our subgroups even though they don't even count toward AYP (For readers who don't understand that-subgroups such as sp ed or free and reduced lunch need 50 students to count in determining whether a school makes AYP). Do you do this at your school? If not, that’s a school problem…Again, have you asked the other 80 schools in LC what data driven programs and interventions they have in place. We do this every year and then look at the current data to determine if our strategies, programs (we have two such programs planned and run by teachers before and after school).
    Should we just quit doing all this extra work?

    What about the schools that are piloting RTI? PBIS? I’m happy that our county PILOTS these research driven programs, collects data and makes the decision on whether or not to expand into other schools. (A cost saving measure indeed).

    Should we let Mrs. Runge know that she shouldn’t be wasting her time developing and grading strong thinking and application tests for her affluent students? Sarcasm not intended. -Add

    ReplyDelete
  8. Forget Hardeback! Are we to believe that Hatrick never once asked why teachers leave? Is it within the realm of reason that over the past three to four years as the school board and administration froze wages and dropped starting pay year after year that they never once thought to see what effects these actions would have on staff and schools?

    How disconnected can you be! These people would never last a day in the real world.

    ReplyDelete
  9. Erv, your criticism of Dr. Hardebeck for not asking employees if they are leaving LCPS for more money until last year is unwarranted for a couple of reasons.

    First she has only been in the position as Assistant Superintendent for Personnel a little over 18 months. Whatever policies were in place prior to then were not her responsibility.

    But in reality I suspect that the reason no one tracked the information prior to last year was that there was no need too. LCPS ranked first, second or third in teacher salaries in the region, depending on education level and experience. It isn’t likely that someone would leave for more money since the salaries were competative with neighboring jurisdictions. But over the last three years that has changed. LCPS now ranks last in the area in starting salaries. Given those circumstances, it now makes sense to ask the question in the exit interview.

    The other thing that has changed over that time period was the selling point to new employees that LCPS had lower class sizes than neighboring school divisions. Over the past two budget cycles class size has risen by an additional student each year making them comparable to neighboring Fairfax. Completive salaries and lower class size were instrumental in attracting teachers to come to LCPS. Don’t underestimate that as a reason many teachers chose to come to work for LCPS.

    In regard to your comment about class size not making a difference, there is extensive research that class size does matter. Class size between 15-20 has the greatest impact on student achievement. Unfortunately that is not realistic from a financial standpoint. But research does show that especially at the lower grade levels, a class size at or below 23/1 does have an impact especially in regard to closing the achievement gap. Currently, LCPS has a 24/1 ratio at the elementary level, a 23.6/1 ratio at the middle school level and a 27.9/1 ration at high school level. While antidotal information, ask any teacher with over 24 students in their class and they will tell you that pupil/teacher ratio does make a difference.

    You continue to talk about outcomes. The only real outcome that matters is student achievement. In that area there is no question that LCPS continues to provide a quality education when compared to national, state and local jurisdictions. And this was accomplished doing a time of unprecedented growth, building over 40 schools in the past 20 years.

    Take a look at the student achievement goals set by the school board eight years ago. In almost every category the 90% SOL pass rate established by them has been achieved.

    You are also way to quick to dismiss the efforts of LCPS teachers. During the same time period, the number of students coming from families living below the poverty level in the county has more than doubled. At the same time the number of ELL student has grown by 25% a year. While all of that was occurring, the achievement gap between minority and white students closed dramatically. For example, the gap six years ago between African American students and White students reading scores were 29% lower. Over that time period the gap has narrowed to 8% while White students’ reading scores increased by 5%. There are few school districts across the country that can show those kinds of gains in student achievement.

    And one last point Erv. The difference between the per pupil expenditures between Fairfax and Loudoun for FY11 is $1,759 per student (Fairfax - $12,597, Loudoun $10,838). If Loudoun were funded at the same level, LCPS would have received an additional $111,344,700 during FY11 . Given that the achievement levels between the two school districts are comparable, I would say LCPS is extremely efficient!

    ReplyDelete
  10. Dr Hatrick and the School Board needed more money and socked it to the HS parents once again(years ago). Now that they can't collect $76 for each AP test, they will find another fee. Those fees had nothing to do with making LCPS more efficient. One thing left off all these posts, is that LCPS has a lot of parents volunteering their time to help out in the classroom and with various other projects. The past budget numbers only show how bloated LCPS was in 2008. The efficiency can be attributed to great kids, teachers and parents in Loudoun.

    ReplyDelete
  11. Great post, Black out II...It's too bad some people are quick to dismiss the incredible accomplishments of LCPS and the benefits a decent and affective (not perfect)school system has had on the LC economy overall.

    ReplyDelete
  12. Great post, Erv! Keep 'em coming.

    The statistics the public hears are cherry-picked and usually only tell half the story. For instance, LCPS may be second to last in starting salaries, but they are tops in benefits.

    Is being second to last that bad? No. Housing and commuting costs are lower in Loudoun, so they don't need to pay as much as inner localities to attract good employees.

    When they talk about staffing levels, yes they have dropped the past few years, but they are still much higher than levels before the great housing boom spending binge. Same with per-pupil spending that used to be less than $7,000 at the beginning of the last decade.

    ReplyDelete
  13. anon-10:07-LC is NOT tops in benefits...They rank fourth out nine school districts in our area.

    ReplyDelete
  14. Simple put a fee on everything you can. Then adjust the fees according to income and amount of children per family. Then a lot of the cost can be burden by the families with children in the schools. And this BS that people that don't have children in school should still pay because previous generations paid for their kids. I say then pare down all the extras the previous generations did not have for their kids in school.

    As far as why counties like China and India and other countries their kids are doing so well in school. It is the parents and I mean just about all parents put far more time into their child's education then american parents do. Just look at the oriental and Indian families here in America those kids are tops in their classes. I see this in my own neighborhood, those kids parents spend far more time with their kids then most of the other parents.

    ReplyDelete
  15. Anon at 6:08 AM - I think you are looking at an old version of the WABE Guide. Try the FY11 guide, which has 8 participating school districts in the DC area.

    But even 4th out of 9 is really good. Why does Loudoun need to be #1 in benefits and salaries? That would cost taxpayers a fortune and leave families with less money to pay for college.

    ReplyDelete
  16. maybe Erv should spend more time collaborating with his team members and planning lessons and less time on placing comments about how poorly he feels the school system is run. No system is perfect but something must be right if our kids are doing so well.
    But what do I know, I am just a parent who wants the best school system for the children....Have not been disappointed thus far.

    ReplyDelete
  17. Oh dear. Anon at 421 actually used the word "oriental" to refer to students.

    ReplyDelete
  18. LCPS needs a budget reduction! LCPS is a perfect example of a bloated government organization that operates like a parasite on taxpayers.

    ReplyDelete
  19. "Erv, your criticism of Dr. Hardebeck for not asking employees if they are leaving LCPS for more money until last year is unwarranted for a couple of reasons.

    First she has only been in the position as Assistant Superintendent for Personnel a little over 18 months. Whatever policies were in place prior to then were not her responsibility."

    To whoever wrote this, please note that according to Mrs. Hardeback's resume she neither has any human resources experience or belongs to any professional HR organizations. I do, and knowing and analyzing termination reasons (voluntary and involuntary) are standard industry best practices. There are no good excuses for ignoring this, by her or her predecessors. I must ask how she is qualified to lead a human resources organization?

    ReplyDelete