This App from Google called Google Refine, looks like it could help allot in all kinds of Research. The thing that interested me was the Web site that he was sifting through. It was from a Site called ProPublica and the article is called Dollars for Docs. This is wat interests me...
Drug companies have long kept secret details of the payments they make to doctors for promoting their drugs. But seven companies have begun posting names and compensation on the Web, some as the result of legal settlements. ProPublica compiled these disclosures, totaling $258 million, into a single database that allows patients to search for their doctor. Receiving payments isn’t necessarily wrong, but it does raise ethical issues.
I'm definitely going to check this site out! But, back to Google Refine and how he used this great App to sift through huge amounts of Data...
Google has just introduced a new product, and this time it's a PC application (with a browser-based UI). It's called Google Refine, and it solves a problem that is enormous for some people: it lets you take massive sets of "messy data" and massage them into shape so that they're uniform, make sense, and can be statistically analyzed.
The video after the jump shows a very good example, which is based on a CSV file exported from a publicly available data source (a government contract system, in this case). The data is very realistic – descriptions are inconsistent (Firm Fixed Price on some rows and FFP on other rows), and even the number formats are inconsistent (you get 0.78 on one row and a number in the millions on another row).
Google Refine lets you very easily hone in on those inconsistencies and fix them in a myriad of ways. This is an important data tool because those heaps of messy data are often public records, which are available but not transparent; being able to quickly analyze them could expose some very interesting patterns and anomalies in the way that public institutions and governments behave.
[Thanks, Yanksy, for the tip!]Go there Read more Articles...
See Video - Google Refine 2.0 - Introduction (1 of 3)