Quote:
Originally Posted by bizaro86
And of course you're right that not all the data the DOE collects is public, which makes sense. That doesn't mean what they do share is irrelevant or useless.
|
We have very different interpretations of useful data. To me, useful data must be complete and valid. When you have incomplete data, and data that is not validated for completeness and accuracy, any study using that data is invalid. You can't attempt to draw conclusions on comparisons when the data is either inaccurate, unavailable, or just doesn't exist. You can blame it on all those damn methods and statistics classes I had to take to earn my terminal degree. They really burned this into my soul which is why I always review studies and their data sets before passing judgement on them. Cherry picked data is the worst, and this data is just that. It is why I say the DOE does not release this information, because the information is not complete, it is inaccurate, and it is not validated. It's useless data unless you are intent on cherry picking data.
It is like trying to draw conclusions about the outcome of a hockey game using only shot counts, and only having shot counts in one end of the ice, and only for one period of the game. What's worse, you also have no idea whether the data was collected by someone who took three bathroom breaks during the game and just how many shots they may have missed or misclassified as shots. Studies and data are only valid when consistent methods are applied and generate consistent outcomes. These data sets are woefully incomplete and there is no validation to the data being studied.
Quote:
|
For example, the columns surrounding student deaths are all private. That seems perfectly logical. However, the columns around student debt by gender are only private at institutions where the sample is small enough to be identifying (and probably not statistically significant anyway).
|
Technically, all of that data is private without having waivers from the students. I think this may be where some of the data is available. Some institutions use broad waivers that students are forced to sign, allowing those institutions to share the information more broadly than the students may understand. The ethics of such waivers, and the subsequent disclosure of private data, is substantial argument across the industry, especially with the increasing data privacy laws coming on line around the globe.
Quote:
|
But luckily, that leaves 4434 programs with information on student debt by gender. Seems like enough data to be useful to me, and on average institutions had a median female student debt of 6% higher than males.
|
Incomplete and unvalidated information. Keep that in mind as you review this information. Completeness and validity matters. That would easily explain some of the things you noted, like the 6% higher debt load for females over males. You look at those numbers and will immediately draw some pretty bad conclusions, even though the rest of the data set would likely easily explain this and why that number is actually probably lower than it really is. The complete data would show there are more women in higher ed, and their completion rate is much higher, meaning they are going to naturally have much higher debt loads. Male students tend to drop out with greater regularity driving their completion stats way down in comparison. A complete data set would allow you to achieve these conclusions and quickly explain these things.
Quote:
|
Do you think the fact that a rabbinical college that graduates three people per year isn't reported on privacy grounds makes the data for large public universities any less relevant?
|
No, the incompleteness of the data from the large institutions makes the data irrelevant.
Quote:
|
And for the relevant income columns, the data is very complete. In fact, for the column "Share of students earning over $25,000/year (threshold earnings) 6 years after entry" there are 4676 data points. That's a lot of programs to draw a conclusion from.
|
Except that the data is not reliable, because schools do not have access to that information unless the student freely gives that data up themselves post graduation. For example, how many surveys from your school have you completed, and how honest were you in those surveys? That information would also have to be carefully tracked and validated, which institutions are incapable of doing. The only interest that can do this is DOE, and only the only mechanism they can do so without student reporting is through the administration of student loans, which is again incomplete data. The mechanism they would have to rely upon is IRS reporting done for students leveraging income-based payment plans, which is again a subset and incomplete.
Quote:
|
But this is a situation (just like Obama's reform attempts) where the academic establishment has a lot to lose from thoughtful reform. Take somewhere like Ohio's Chatfield College - only 20.9% of their students are earning more thank 25k six years later. I bet it would hurt recruiting if that was widely known. University of Arkansas at Pine Bluff comes in at 38%. Even the big state school are often only 70% - University of South Florida is 68.5%. So for that large state school more than 1 in 5 people who enter it earn 25k or less 6 years later. That's about the cut-off for SNAP (fka Food Stamps).
|
Which drives a number of questions. How was this determined? Where did the data come from? What percentage of students provided this data? How was the data validated? Again, institutions do not have access to this data, nor do they track this information from graduates in any consistent manner. The data is not going to be complete and is not going to be validated. If these numbers were true, the vast majority of the nation would be on food stamps.
Quote:
|
As to the idea that self reported incomes wouldn't be perfectly accurate, I agree with that. I also think there's no particular reason to think the skew would be in any one direction (it seems likely errors would be in both directions at the same rate and magnitude). Even if students persistently either over or under reported their income it seems incredibly unlikely that error would persistently favor one type of program or institution across a sample size of thousands of data points. So I agree the data isn't perfect, but that's a far cry from your original claim that it didn't exist and your current claim that it's useless.
|
The data is garbage. When you have a data set that has almost 151,000 null fields, and over 19,000,000 privacy suppressed fields, that is a lot of information that makes the data useless. You're pointing to 4,500 data points and trying to make conclusions. The data to do so isn't there. I stand by my comment because I know how the data is collected and the data that is missing. The data is not made available and is useless. Privacy laws and disconnected systems prevent real useful data from being available on this. The feds have access to it, but they cannot legally release this data. Trying to make conclusions from the self reported data is unreliable.
Now, there are many reasons why students do not report information back to their school with this information. Privacy being the top one. Dissatisfaction being up there as well. Just having moved on with their lives and not caring anymore is also up there. I think that most people just won't report this information, or give accurate information, as it is none of the school's business.
But I do know of another, and it will make you laugh. Under reporting is a smart strategy to prevent the school from pestering you in the future with fund raising requests. If you are a low income earner the Development office won't bother you with requests for giving. If you are a high income earner they will continually pester you for money. So if you report low income you get off the target list. It's a pretty smart dodge to stay off the Development Office's target list. I don't think this is a huge number, but one that I have heard many times in cabinet meetings from our Development Office.
Quote:
|
What do you think of Obama's plan to tie the availability of Federal student aid to a college scorecard including outcomes? This was implemented to a certain level for for-profit institutions, but I think there are plenty of terrible non-profit ones around as well. Not to overly pick on Chatfield (mentioned above) as I doubt they're the worst, but they are a Roman Catholic institution offering non-theological degrees (not for profit). It seems like most of their students would probably be better off attending elsewhere (or not at all) and I don't see any reason why the government should be supporting such poor outcomes.
|
I liked Obama's proposal. Higher Ed needs to have spending reigned in, and it has to be done systemically. But there is the problem. State schools can be mandated to follow initiatives like this, but private schools can't. Those private schools really drive the industry and everyone is chasing their success. If you can't control the flow of money into these private schools then the public schools are screwed too. I think the best way to address this concern is systemic control of student funding, and only the government can do that. But the problem there is the government outsources this mechanism and it is also profit driven. So the problem is never able to be controlled.
