randomosity

I attended a very informative and thought provoking presentation by Peter Baskerville at the University of Guelph today. He postulates that the
shift in wealth from men to women during the period 1860 - 1930 was of similar magnitude to that of the land grab by European settlers from native Canadians. His presentation was based on material from his forthcoming book “Silent Revolution: Wealth and Gender in Canada, 1860-1930.” Baskerville’s work in this book, as in past, rests on his impressive use of cutting edge quantitative analysis and synthesis of census data with other official records. His impressive record of articles, books and edited volumes has shed new light on the life of ordinary Canadians during the nineteenth and early twentieth centuries.
Read the complete article… »

I have been of late explores various means for the automated longitudinal matching of census manuscript records. Its a huge challenge and I seem to have spent as much time identifying potential problems as opposed to identifying potential solutions. This is not say I haven’t pondered a couple solutions, but the list of challenges remains much longer and seems to be growing much faster - but, all this means is a more challenging research problem, demanding some innovation in methodology. Fun!

But there is a paradigm shift happening. One that I have been participating in, and certainly embrace, but am seldom always cognizant of. The idea of online collaboration continues to permeate more and more of our everyday tasks. Emerging from specialized research objectives such as the SETI@Home initiative, which sought to use excess personal computing capacity distributed around the world, to other efforts today that take advntage not only of excess processor cycles to the idea of carrying out manual tasks through engagement of the masses in specific tasks.

I started playing with the Google Image identification programme a few months back. If you haven’t tried it, it basically involves matching you with a random online user and you spend 90 seconds typing in words to describe a picture displayed to both users. You quickly type words that come to mind until both users type in the same word, at which point the engine accepts that that word is likely to be a relevant descriptor. The key to participation is that the exercise if fun, fast and you can hop on at anytime and given the global scope, you will quickly be paired with an online user. Moreover, you have the small satisfaction of being part of a bigger exercise of improving the descriptors attached to Google’s image search repository. This little ‘game’ also clearly illustrates one of the downsides of Google’s repository, as these descriptors are determined through a process which renders them simple rather than more specialized. as I ‘play’ I realize that I may recognize the image as a particular movie poster, but also think that my online partner may not catch the subtleties, so I may resort to simply choosing a predominant colour as a suggested word, rather than the name of the movie or say an actor in the movie. As a result I choose the more obvious descriptor word to encourage faster match. The objective in the Google match is to match words for the highest number of images during the 90 second period, which may not achieve the best descriptions. However, the process does deliver some basic descriptions terms that an automated process would miss. The key is making it fun for the participants.

Down this same vein, Kris Inwood pointed me at a census initiative, Automated Genealogy. Working down this same premise of trying to funify a process requiring mass user intervention, at Automated Genealogy, the site is a meeting point for genealogists to signup for and manually enter into a database manuscript census records. The hope here is to engage that vast army of genealogists out there to contribute time to help their fellow genealogists and have access to records which benefit their own research efforts. Collaboration at its best. Additionally they have begun a similar process to match Canadian manuscript census records between the 1901 and 1911 censuses. This is the same task that I have been ruminating over developing an automated process for. At AG they are using automated means to do simple matching and then allowing users to refine the match where human discretion is required. This is a clever approach to a real world research problem. As to progress, the published results indicate that they have transcribed 93.15% of the entire Canadian census for 1911 and 99.99% of the 1901 census with 55.15% of the proofing carried out on this one.

This is a great example of this emerging trend to mobilize individual efforts en masse to assist with processes that in the past would have been carried out by a small group of specialized researchers. Both processes recognize that tasks can be divided and appropriate and different resources applied to varying stages. Mass collaboration on simple tasks made fun!

On Tuesday, I had the pleasure of meeting with Kris Inwood, Director of the 1891 Census Project at the University of Guelph along with his staff at a review of this exciting project.

Census project staff have been entering data since 2002 and as of last Friday have completed the data entry phase. They have compiled a database comprising 328,000 records which represents a 5% sample of the entire population of Canada in 1891. They have oversampled in certain urban areas as well as in the west of Canada to 10%. There is also a 100% capture of group quarters (households with more than 30 residents indicated in the manuscript census records). The next step in the project is to begin coding columns such as religion and occupation to allow for systematic use by researchers.

Over the life of the project participants have also been conducting research on their own interests using census data. A number have completed very interesting papers examining topics such as the character and nature of the enumerators, the foibles of the enumeration process, methodology involved in locating aboriginal persons in the census and a survey of contemporary newspaper coverage of the census itself.

Additionally impressive, many of the participants have contributed to a series mini-biographies of individuals and families in the census which will hopefully be shared via the census website. These papers illuminate the human side of manuscript census records and they also provide very useful case studies demonstrating how census manuscript data can be used in a variety of research contexts.

Kris suggests that they are very close to being able to provide researchers with the opportunity to begin to use this data outside the project and avenues are now being explored to provide systematic dissemination of the dataset.