Category Archives: Survey of Irish Settlers

Materials from A-632 that pertain to questions addressed to the Irish Settlers of 1823 & 1825 and their answers.

Quality Control for Crowd-sourced Transcription

Whenever I talk about crowd-sourced transcription–actually whenever Italk about crowdsourced
–the first question people ask is aboutaccuracy. Nobody trusts the public add to an institution’s data/meta-data,nor especially to correct it. However, quality control over data entry is awell-explored problem, and while I’m not familiar with the literaturefrom industry regarding commercial approaches, I’d like to offer thesystems I’ve seen implemented in the kinds of volunteer transcriptionprojects I follow.
(Note: the terminology is my own, and may be non-standard.)


Single-track methods
(mainly employed with large, prosy textthat is difficult to compare against independent transcriptions of the same text). In these methods, all changes and corrections aremade to a single transcription which originated with a volunteerand is modified thereafter. There no parallel/alternate transcriptionto compare against.

Open-ended community revision:
This is the method thatWikipedia uses, and it’s the strategy I’ve followed inFromThePage.In this method, users may continue to changethe text of a transcription forever. Because all changes arelogged–with a pointer of some sort to the user who loggedthem–vandalism or edits which are not in good faith may bereverted to a known-good state easily. This is in keeping withthe digital humanities principle of “no final version.” In myown projects, I’ve seen edits made to a transcription twodecades after the initial version, and those changes were
indeed correct. (Who knew that “drugget” was a coarse fabricused for covering tobacco plant-beds?) Furthermore, Ibelieve that there is no reason other than the cost of implementation why any of the methods below which operatefrom the “final version” mind-set should not allow errorreports against their “published” form.2.

Fixed-term community revision:
Early versions of bothTransribeBentham andScripto followed this model, andwhile I’m not sure if either of them still do, it does seem toappeal to traditional documentary editing projects that areincorporating crowdsourcing as a valuable initial input to aproject while wishing to retain ultimate control over the”final version”. In this model, wiki-like systems are used togather the inital data, with periodic review by experts. Once atranscription reaches an acceptable status (deemed so by theexperts), it is locked to further community edits and thetranscription is “published” to a more traditional medium likea CMS or a print edition.3.

Community-controlled revision

any other project I’m aware of.) If a transcription is rejected,it may be either returned to the submitter for correction orcorrected by the expert and published in corrected form.B.

Multi-track methods
(mainly employed with easily-isolated,structured records like census entries or ship’s log books). In all of these cases, the same image is presented to different users to betranscribed from scratch. The data thus collected is comparedprogrammatically on the assumption that two correct transcriptionswill agree with each other and may be assumed to be valid. If thetwo transcriptions disagree with each other, however, one of themmust be in error, so some kind of programmatic or human expertintervention is needed. It should be noted that all of thesemethodologys are technically “blind” n-way keying, as thevolunteers are unaware of each other’s contributions and do notknow whether they are interpreting the data for the first time orcontributing a duplicate entry.6.

Triple-keying with voting:
This is the method that theZooniverseOldWeather team uses. Originally theOldWeather team collected the same information in tendifferent independent tracks, entered by users who wereunaware of each other’s contributions: blind, ten-way keying.The assumption was that majority reading would be thecorrect one, so essentially this is a voting system. After someanalysis it was determined that the quality of three-waykeying was indistinguishable from that of ten-way keying, sothe system was modified to a less-skeptical algorithm, savingvolunteer effort. If I understand correctly, the same kind of voting methodology is used by ReCAPTCHA for its OCRcorrection, which allowed its exploitation by 4chan.


Double-keying with expert reconciliation:
In this system,the same entry is shown to two different volunteers, and if their submissions do not agree it is passed to an expert forreconciliation. This requires a second level of correctionsoftware capable of displaying the original image along withboth submitted transcriptions. If I recall my fellow panelistDavid Klevan’s WebWise presentation correctly, this systemis used by the Holocaust Museum for one of theircrowdsourcing projects.8.

Double-keying with emergent community-expertreconciliation:
This method is almost identical to theprevious one, with one important exception. The experts whoreconcile divergent transcriptions are themselves volunteers — volunteers who have been promoted to from transcribers toreconcilers through an algorithm. If a user has submitted acertain (large) number of transcriptions, and if thosetranscriptions have either 1) matched their counterpart’ssubmission, or 2) been deemed correct by the reconcilerwhen they are in conflict with their counterpart’stranscription, then the user is automatically promoted. Afterpromotion, they are able to choose their volunteer activityfrom either the queue of images to be transcribed or thequeue of conflicting transcriptions to be reconciled. This isthe system used by FamilySearch Indexing, and its emergentnature makes it a particularly scalable solution for qualitycontrol.9.

Double-keying with N-keyed run-off votes:
Nobodyactually does this that I’m aware of, but I think it might becost-effective. If the initial set of two volunteer submissionsdon’t agree, rather than submit the argument to an expert, re-

queue the transcription to new volunteers. I’m not sure whatthe right number is here — perhaps only a single tie-breakervote, but perhaps three new volunteers to provide anoverwhelming consensus against the original readings. If thisis indecisive, why not re-submit the transcription again to aneven larger group? Obviously this requires some limits, orelse the whole thing could spiral into an infinite loop inwhich your entire pool of volunteers are arguing with eachother about the reading of a single entry that is trulyindecipherable. However, I think it has some promise as itmay have the same scalability benefits of the previousmethod without needing the complex promotion algorithmnor the reconciliation UI.
Some things are simply not knowable. It is hard to evaluate theeffectiveness of quality control seriously without taking into account thepossibility that volunteer contributors may be correct and experts may bewrong, nor more importantly that some images are simply illegibleregardless of the paleographic expertise of the transcriber. TheZooniverse team is now exploring ways for volunteers to correct errorsmade not by transcribers but rather by the midshipmen of the watch whorecorded the original entries a century ago. They realize that a mistaken”E” for “W” in a longitude record may be more amenable to correctionthan a truly illegible entry. Not all errors are made by the “crowd”, after all.

More on the Survey of Robinson’s Irish Settlers – 1828

We have located two primary sources of information about the survey of Irish settlers in the Bathurst and Newcastle Districts of Upper Canada in 1828:

The British Government’s Explanation of the Survey to the Irish Settlers

A printed explanation of the British government’s intent in surveying the Irish settlers is included, along with a blank copy of the survey itself, in the Peter Robinson Fonds in the Peterborough Musuem and Archives. The explanation was clearly meant to be shared with the Irish settlers by whomever was administering the survey:1

The Emigrations from Ireland to Upper Canada, in 1823 and 1825, under the direction of the Government are believed to have been eminently successful, so far as regards the comfortable settlement of the emigrant, and the accomplishing that happy change in his condition, which the government had in view.


Besides the various accounts which have reached the government through official channels, and by other means, the private letters of many of the settlers to their friends in Ireland, have furnished the most pleasing and convincing evidence of their prosperity, and of their thankfulness for all that has been done for them.


By their general good conduct, they have made the best return in their power for the exertions made in their behalf, and the government has every reason to be fully convinced of the complete success of these experiences, and to be satisfied with the results which have followed from them.


They have been so far encouraged by these results as to think, that provision might be made in the same manner for conveying many thousands of the poor population of Ireland to Canada, and that the benefit which such emigrants would derive from it, would be so great, and so certain that they might be fairly expected to repay, within a reasonable period, the money necessarily advanced by the government for settling them in Canada.


With a view to obtain evidence and to form correct opinions as to the probably success of such a scheme, a committee of the house of Commons was formed in 1827, who made very extensive inquiries upon the subject, and were convinced from those inquiries that the measure might be safely pursued upon such a footing to a great extent.


But it should be known that in Ireland great exertions are made to discountenance such a measure, and great pains are taken there to render the population unfavourable to the idea of relief from colonization, and to characterize their settlement in Upper Canada, as a system of transportation and banishment, attended with privations and misery, and to represent the experimental emigrations which have taken place as unsuccessful.


If these representations and opinions are true, the grounds of them should be so plainly and openly made known, that there can be no farther misapprehension on the subject, and that the government may not proceed further in a plan, which in that case, would be hurtful and not beneficial.

If they are unfounded, it is necessary that they should be shown to be so, from the best evidence that can possibly be given.


With that view you, as one of the persons whom the government has sent to Canada, and who have been long enough there to know, whether you have been benefited by the change, or not, are now informed that efforts are made in Ireland by many persons to dissuade your countrymen from emigrating, by describing it as an injury and an insult, instead of a benefit; while, on the other hand, to those who assisted you in emigrating, it appears so far otherwise that they seem convinced, that a pauper emigrant head of a family, consisting of a man, a woman and three children, would be enabled, without any real difficulty, to pay in the produced of the country by annual instalments, progressively increasing as his means increased, such a sum as would at last cancel the debt of £60, which he would owe the government for locating him and his family, and providing him with sufficient necessaries until he could subsist himself; supposing the sum of £60 to be enough for that purpose, and that by removing him to Canada upon such terms, instead of leaving him in Ireland, they would be doing him a very great favour.


To give the government the benefit of your candid and sincere opinion on this point, the following queries are put to you, to which it is wished that you should give an honest answer, not allowing yourself to be swayed by any other feeling than a desire to speak the plain truth according to the best of your judgment, whether your opinion does or does not agree with the view of the question taken by the government.


Horton’s Analysis of the Irish Settlers’ Responses

In his monograph, Ireland and Canada (1839), Horton shares with the public the views of Charles Rubidge, who had been, by that time, “a settler in Canada for the space of nearly twenty years.” With an official forum no longer at his disposal, Horton contrives to elicit Mr. Rubidge’s views “by proposing to him that course of ordinary questions which I conceive would be put to him in a Committee of the House of Commons, by a Member anxious to examine him in chief ….”2 Horton’s examination of Mr. Rubidge included the following exchange regarding the survey of the Irish settlers in 1828:

In June, 1828, more than ten years ago, the following questions were put at my request to the heads of 180 families, who formed part of the emigrations of 1823 and 1825. I have added a summary of their answers; and my question to you is, whether, after a lapse of ten years, they would return answers of a similar nature?


  1. From whence did you emigrate to Upper Canada, and when?
  2. What was your trade or occupation at home? And what were your circumstances when you embarked?
  3. Did you come out independently of any public assistance?
  4. If you were assisted by the public, what assistance was given you, and under whose superintendence were you?
  5. Had you any money when you came out, and how much?
  6. What are your present circumstances, as to house and other buildings, lands cleared and fenced and farming stock?
  7. What family had you with you when you embarked?
  8. Did any of your family die on the passage to Quebec; and, if so, how many?
  9. What state of health were they in during the last year?
  10. To what value had you produce or live stock to dispose of in the last year, above what you required for your family?
  11. On what kind of provisions does your family usually subsist?
  12. Are you pleased with your situation in Upper Canada?
  13. Have your comfort and happiness been increased by coming to Upper Canada?
  14. Would you advise any of your friends in the Country you left, whose situation there is the same as yours was, to come out to Upper Canada upon the same terms that you did?
  15. Suppose the government had furnished you and your family with a passage out, paid your expenses to your lands, gives you 100 acres free of expense, provisions for a year, and the necessary farming utensils, and that this was done upon the condition that you should repay the sum advanced by annual instalments, beginning to pay at the end of [      ] years, and after you had been settled, and paying [      ] pounds in each year after, until the whole was paid up, would it have been in your power to make those payments?
  16. Knowing Upper Canada as you do now, would you think it advisable for a head of a family in Ireland, who is now poor, and without employment, to accept of such terms?
  17. Would it be better for him to receive from government, after landing in Quebec, £60, or whatever may be necessary for taking himself and his family to his land, finding him provisions for a year, and farming utensils, upon the condition of his repaying to the government, the amount so advanced to him, either in money or the produce of his land, or to be merely landed at Quebec, and afterwards to depend upon his own exertions for establishing himself and family?

The answers given to several of these questions, of course, vary considerably in the cases of the different settlers. Nearly all of them state their circumstances in Ireland to have been very bad; and the greater number had absolutely no money at the time of their embarkation. With respect to their present situation, their satisfaction is in general expressed very decidedly, and in several instances with a remarkable appearance of cordial and grateful feeling. Their answers to Questions 12 are very various, some having disposed, during the preceding year, of produce to the value of £3., £4., £5., £6., and in one case £12., while others had raised only what was required for the consumption of their families. Some of this latter class assign, as their reasons for their not having disposed of any surplus produce, the unfavourable season of the preceding year, the distance of markets, or some peculiarity of their own circumstances. The answers to Question 13 are equally various, depending probably on the previous habits of the parties, the produce and the stock which they had raised, &c. The articles of food most frequently mentioned are pork (sometimes beef), flour, Indian meal, potatoes, milk, and butter. One settler answers, “variety and plenty;” and another, “the best that Upper Canada can afford.” Only one man complains that the produce of his farm was not enough to supply the wants (including clothing) of his family, which consisted of nine children; but even he concurs with the other settlers, in stating that his “comfort and happiness have been increased by coming to Upper Canada;” and to Question 16 (whether he would advise poor persons in Ireland to accept of such assistance as had been given to him, and to emigrate to Upper Canada?) his answer is – “I would indeed.” The answers to this question are without exception in the affirmative, and some of the settlers add, that they have already by letter given such advice to their friends. I wish particularly, however, to call the attention of those who may read this statement, to the three concluding questions, which relate to the repayment, on the part of the settlers, of the expense incurred by government in their location. In one instance those three questions are returned unanswered, and in two or three others they appear to have been misunderstood, and the answers are consequently unintelligible. The one complaining individual already mentioned, having filled up the first blank in Question 17 with the figure 5, answers that question in the negative; and to Question 18 he replies, — “I would; but if he had a large family, I think he would not be able to pay much at the expiration of five years.” Another would postpone the commencement of repayment until ten years after the location of the settler. Three others think that repayment would be practicable after five years, if not prevented by casualties. All the other settlers express a positive opinion, that progressive repayment, in produce, could be effected; the majority fixing five years as the period of commencement and the others leaving the period in blank. Their opinions vary with respect to the amount of instalment which should be fixed, with a view to the convenience of the settler. The answers to the two last questions are unanimous in favour of the acceptance by an emigrant of assistance from government, upon condition of progressive repayment in produce, in preference to a reliance on his own unassisted exertions. In several instances an opinion is added, as to the comparative value of a loan in money, or an equivalent supply of necessary articles.


I have thus stated at length the general results of the answers returned to these questions, with the view of conveying as fairly as possible, the impressions which would be produced by an examination in detail of these 180 documents. The variety of the answers, both in substance and language, sufficiently show that they are the free expression of the judgment and feeling of the parties. All of these settlers had been resident in the colony three years, and many of them five years.

Horton added (in a footnote):

If a reasonable scepticism were to be expressed from any parties deserving attention, as to the accuracy of the general results of the answers returned to the questions which have been put to the emigrants, I should not feel the slightest objection to giving, in the minutest detail, the 180 documents to which reference has been made.

To which, Mr. Rubidge finally responded:

You have asked me the question, whether in my opinion those 180 Irish settlers would, after the lapse of ten years, which has now taken place, return answers of the same tenor as those they returned in 1828? To that question I answer, that in my judgment they unquestionably would return the same answers, or answers equally satisfactory. I should mention that I have had an opportunity of seeing those 180 answers in detail, by which I am more confirmed in my opinion.[3. Ibid., pp. 68 - 71.]


 A Local Controversy Resolved

The late historian, Howard Pammett, is rightly acknowledged for his numerous contributions to our understanding of early life in Upper Canada, and the settlement of townships throughout the Peterborough region in particular. Yet Mr. Pammett was liable to the same missteps as other ground-breaking researchers — and his remarks about Horton’s survey of the Irish Settlers were especially unfortunate:

Horton made several other attempts to arouse interest, and to prove the success of the emigrants of 1823 and 1825, after the Emigration Committee Reports of 1826-7 were ignored. In 1828 a questionnaire was sent to 180 heads of families in the Ottawa and Peterborough districts. Horton asserted that these were all heads of families “who formed part of the emigrations of 1823 and 1825,” but an examination of the 80 original answered questionnaires in the Peterborough Library Collection proves that the majority were answered by English and Scots residing along the lakefront and elsewhere in the district, who had come out at various times since 1815. From these Horton deduced the prosperity of the Irish emigrants, but both premises and conclusions were based on unreliable information. His conclusion that: “The answers to the two last questions are unanimous in favour of the acceptance by an emigrant of assistance from Government, upon condition of progressive repayment in produce, in preference to a reliance upon his own unassisted exertions” sounded the keynote to his whole policy of assisted emigration, and was no doubt a wise policy if it could be carefully applied, but it did harm rather than good to attempt to prove it by subversions and distortions of the truth.3

With the recent discovery of 180 responses from the Irish settlers in the Sir Robert Wilmot Horton Fonds, it seems that Horton’s survey had been administered to 260 individuals or so – 180 Irish settlers, whose responses were forwarded to Horton in England, and 80 Scots and English settlers, whose responses were held back in Upper Canada, and eventually became part of the Peter Robinson Fonds in the Peterborough Library Collection.

  1. The one page explanation and two page survey were located at the end of Section 10 – Emigrants Embarked at Cove, 1825 in the microfilm copy of the Peter Robinson Fonds held at the Archives of Ontario.
  2. Sir Robert Wilmot Horton, Ireland and Canada; Supported by Local Evidence, p. 36, 1839.
  3. H. T. Pammett, “Assisted Emigration from Ireland to Upper Canada under Peter Robinson in 1825,” Papers and Records, Ontario Historical Society, vol. xxxi, pp. 211 – 212, 1936.