Phd Dissertation How Many Words Is This

Note: Please see the update to this blog!

The best part about writing a dissertation is finding clever ways to procrastinate. The motivation for this blog comes from one of the more creative ways I’ve found to keep myself from writing. I’ve posted about data mining in the past and this post follows up on those ideas using a topic that is relevant to anyone that has ever considered getting, or has successfully completed, their PhD.

I think a major deterrent that keeps people away from graduate school is the requirement to write a dissertation or thesis. One often hears horror stories of the excessive page lengths that are expected. However, most don’t realize that dissertations are filled with lots of white space, e.g., pages are one-sided, lines are double-spaced, and the author can put any material they want in appendices. The actual written portion may only account for less than 50% of the page length. A single chapter may be 30-40 pages in length, whereas the same chapter published in the primary literature may only be 10 or so pages long in a journal. Regardless, students (myself included) tend to fixate on the ‘appropriate’ page length for a dissertation, as if it’s some sort of measure of how much work you’ve done to get your degree. Any professor will tell you that page length is not a good indicator of the quality of your work. Regardless, I feel that some general page length goal should be established prior to writing. This length could be a minimum to ensure you put forth enough effort, or an upper limit to ensure you aren’t too excessive on extraneous details.

It’s debatable as to what, if anything, page length indicates about the quality of one’s work. One could argue that it indicates absolutely nothing. My advisor once told me about a student in Chemistry that produced a dissertation that was less than five pages, and included nothing more than a molecular equation that illustrated the primary findings of the research. I’ve heard of other advisors that strongly discourage students from creating lengthy dissertations. Like any indicator, page length provides information that may or may not be useful. However, I guarantee that almost every graduate student has thought about an appropriate page length on at least one occasion during their education.

The University of Minnesota library system has been maintaining electronic dissertations since 2007 in their Digital Conservancy website. These digital archives represent an excellent opportunity for data mining. I’ve developed a data scraper that gathers information on student dissertations, such as page length, year and month of graduation, major, and primary advisor. Unfortunately, the code will not work unless you are signed in to the University of Minnesota library system. I’ll try my best to explain what the code does so others can use it to gather data on their own. I’ll also provide some figures showing some relevant data about dissertations. Obviously, this sample is not representative of all institutions or time periods, so extrapolation may be unwise. I also won’t be providing any of the raw data, since it isn’t meant to be accessible for those outside of the University system.

I’ll first show the code to get the raw data for each author. The code returns a list with two elements for each author. The first element has the permanent and unique URL for each author’s data and the second element contains a character string with relevant data to be parsed.

#import package require(XML) #starting URL to search<-'' #output object dat<-list() #stopping criteria for search loop stp.txt<-'2536-2536 of 2536.' str.chk<-'foo' #initiate search loop while(!grepl(stp.txt,str.chk)){ html<-htmlTreeParse(,useInternalNodes=T) str.chk<-xpathSApply(html,'//p',xmlValue)[3] names.tmp<-xpathSApply(html, "//table", xmlValue)[10] names.tmp<-gsub("^\\s+", "",strsplit(names.tmp,'\n')[[1]]) names.tmp<-names.tmp[nchar(names.tmp)>0] url.txt<-strsplit(names.tmp,', ') url.txt<-lapply( url.txt, function(x){ cat(x,'\n') flush.console() #get permanent handle url.tmp<-gsub(' ','+',x) url.tmp<-paste( '', paste(url.tmp,collapse='%2C+'), sep='' ) html.tmp<-readLines(url.tmp) str.tmp<-rev(html.tmp[grep('handle',html.tmp)])[1] str.tmp<-strsplit(str.tmp,'\"')[[1]] str.tmp<-str.tmp[grep('handle',str.tmp)] #permanent URL #parse permanent handle perm.tmp<-htmlTreeParse( paste('',str.tmp,sep=''),useInternalNodes=T ) perm.tmp<-xpathSApply(perm.tmp, "//td", xmlValue) perm.tmp<-perm.tmp[grep('Major|pages',perm.tmp)] perm.tmp<-c(str.tmp,rev(perm.tmp)[1]) } ) #append data to list, will contain some duplicates dat<-c(dat,url.txt) #reinitiate url search for next iteration<-strsplit(rev(names.tmp)[1],', ')[[1]]<-gsub(' ','+',<-paste( '', paste(,collapse='%2C+'), sep='' ) } #remove duplicates dat<-unique(dat)

The basic approach is to use functions in the package to import and parse raw HTML from the web pages on the Digital Conservancy. This raw HTML is then further parsed using some of the base functions in R, such as and . The tricky part is to find the permanent URL for each student that contains the relevant information. I used the ‘browse by author’ search page as a starting point. Each ‘browse by author’ page contains links to 21 individuals. The code first imports the HTML, finds the permanent URL for each author, reads the HTML for each permanent URL, finds the relevant data for each dissertation, then continues with the next page of 21 authors. The loop stops once all records are imported.

The important part is to identify the format of each URL so the code knows where to look and where to re-initiate each search. For example, each author has a permanent URL that has the basic form plus ‘handle/12345’, where the last five digits are unique to each author (although the number of digits varied). Once the raw HTML is read in for each page of 21 authors, the code has to find text where the word ‘handle’ appears and then save the following digits to the output object. The permanent URL for each student is then accessed and parsed. The important piece of information for each student takes the following form:

This code is found by searching the HTML for words like ‘Major’ or ‘pages’ after parsing the permanent URL by table cells (using the <td></td> tags). This chunk of text is then saved to the output object for additional parsing.

After the online data were obtained, the following code was used to identify page length, major, month of completion, year of completion, and advisor for each character string for each student. It looks messy but it’s designed to identify the data while handling as many exceptions as I was willing to incorporate into the parsing mechanism. It’s really nothing more than repeated calls to using appropriate search terms to subset the character string.

#function for parsing text from website get.txt<-function({ #separate string by spaces<-strsplit(gsub(',',' ',,fixed=T),' ')[[1]]<-gsub('.','',,fixed=T) #get page number pages<[grep('page',[1]-1] if(grepl('appendices|appendix|:',pages)) pages<-NA #get major, exception for error if(class(try({ major<[c( grep(':|;',[1]:(grep(':|;',[2]-1) )] major<-gsub('.','',gsub('Major|Mayor|;|:','',major),fixed=T) major<-paste(major[nchar(major)>0],collapse=' ') }))=='try-error') major<-NA #get year of graduation yrs<-seq(2006,2013) yr<[grep(paste(yrs,collapse='|'),[1]] yr<-gsub('Major|:','',yr) if(!length(yr)>0) yr<-NA #get month of graduation months<-c('January','February','March','April','May','June','July','August', 'September','October','November','December') month<[grep(paste(months,collapse='|'),[1]] month<-gsub('dissertation|dissertatation|\r\n|:','',month) if(!length(month)>0) month<-NA #get advisor, exception for error if(class(try({ advis<[(grep('Advis','computer',] advis<-paste(advis,collapse=' ') }))=='try-error') advis<-NA #output text c(pages,major,yr,month,advis) } #get data using function, ran on 'dat' check.pgs<'rbind', lapply(dat,function(x){ cat(x[1],'\n') flush.console() c(x[1],get.txt(x[2]))}) ) #convert to dataframe check.pgs<,sringsAsFactors=F) names(check.pgs)<-c('handle','pages','major','yr','month','advis') #reformat some vectors for analysis check.pgs$pages<-as.numeric(as.character(check.pgs$pages)) check.pgs<-na.omit(check.pgs) months<-c('January','February','March','April','May','June','July','August', 'September','October','November','December') check.pgs$month<-factor(check.pgs$month,months,months) check.pgs$major<-tolower(check.pgs$major)

The section of the code that begins with takes the online data (stored as on my machine) and applies the function to identify the relevant information. The resulting text is converted to a data frame and some minor reworkings are applied to convert some vectors to numeric or factor values. Now the data are analyzed using the object.

The data contained 2,536 records for students that completed their dissertations since 2007. The range was incredibly variable (minimum of 21 pages, maximum of 2002), but most dissertations were around 100 to 200 pages.

Interestingly, a lot of students graduated in August just prior to the fall semester. As expected, spikes in defense dates were also observed in December and May at the ends of the fall and spring semesters.

The top four majors with the most dissertations on record were (in descending order) educational policy and administration, electrical engineering, educational psychology, and psychology.

I’ve selected the top fifty majors with the highest number of dissertations and created boxplots to show relative distributions. Not many differences are observed among the majors, although some exceptions are apparent. Economics, mathematics, and biostatistics had the lowest median page lengths, whereas anthropology, history, and political science had the highest median page lengths. This distinction makes sense given the nature of the disciplines.

I’ve also completed a count of number of students per advisor. The maximum number of students that completed their dissertations for a single advisor since 2007 was eight. Anyhow, I’ve satiated my curiosity on this topic so it’s probably best that I actually work on my own dissertation rather than continue blogging. For those interested, the below code was used to create the plots.

###### #plot summary of data require(ggplot2) mean.val<-round(mean(check.pgs$pages)) med.val<-median(check.pgs$pages) sd.val<-round(sd(check.pgs$pages)) rang.val<-range(check.pgs$pages) txt.val<-paste('mean = ',mean.val,'\nmed = ',med.val,'\nsd = ',sd.val, '\nmax = ',rang.val[2],'\nmin = ', rang.val[1],sep='') #histogram for all hist.dat<-ggplot(check.pgs,aes(x=pages)) pdf('C:/Users/Marcus/Desktop/hist_all.pdf',width=7,height=5) hist.dat + geom_histogram(aes(fill=..count..),binwidth=10) + scale_fill_gradient("Count", low = "blue", high = "green") + xlim(0, 500) + geom_text(aes(x=400,y=100,label=txt.val)) #barplot by month<-ggplot(check.pgs,aes(x=month,fill=..count..)) pdf('C:/Users/Marcus/Desktop/month_bar.pdf',width=10,height=5.5) + geom_bar() + scale_fill_gradient("Count", low = "blue", high = "green") ###### #histogram by most popular majors #sort by number of dissertations by major get.grps<-list(c(1:4),c(5:8))#,c(9:12),c(13:16)) for(val in 1:length(get.grps)){ pop.maj<-names(sort(table(check.pgs$major),decreasing=T)[get.grps[[val]]]) pop.maj<-check.pgs[check.pgs$major %in% pop.maj,]<-aggregate(pop.maj$pages,list(pop.maj$major),function(x) round(median(x))) pop.n<-aggregate(pop.maj$pages,list(pop.maj$major),length) hist.maj<-ggplot(pop.maj, aes(x=pages)) hist.maj<-hist.maj + geom_histogram(aes(fill = ..count..), binwidth=10) hist.maj<-hist.maj + facet_wrap(~major,nrow=2,ncol=2) + xlim(0, 500) + scale_fill_gradient("Count", low = "blue", high = "green") y.txt<-mean(ggplot_build(hist.maj)$panel$ranges[[1]]$y.range) txt.dat<-data.frame( x=rep(450,4), y=rep(y.txt,4),$Group.1, lab=paste('med =',$x,'\nn =',pop.n$x,sep=' ') ) hist.maj<-hist.maj + geom_text(data=txt.dat, aes(x=x,y=y,label=lab))<-paste('C:/Users/Marcus/Desktop/group_hist',val,'.pdf',sep='') pdf(,width=9,height=7) print(hist.maj) } ###### #boxplots of data for fifty most popular majors pop.maj<-names(sort(table(check.pgs$major),decreasing=T)[1:50]) pop.maj<-check.pgs[check.pgs$major %in% pop.maj,] pdf('C:/Users/Marcus/Desktop/pop_box.pdf',width=11,height=9) box.maj<-ggplot(pop.maj, aes(factor(major), pages, fill=pop.maj$major)) box.maj<-box.maj + geom_boxplot(lwd=0.5) + ylim(0,500) + coord_flip() box.maj + theme(legend.position = "none", axis.title.y=element_blank())

Update: By popular request, I’ve redone the boxplot summary with major sorted by median page length.

Like this:



Posted on by beckmw. This entry was posted in R, Uncategorized and tagged data mining, dissertation, html, r, xml. Bookmark the permalink.

Many PhD students are now in the final throes of writing their thesis. Turning years of research into a single, coherent piece of work can be tough, so we asked for tips from supervisors and recent PhD graduates. We were inundated with tweets and emails and @AcademiaObscura helpfully created a Storify of the tweets. Below is a selection of the best tips.

1) Make sure you meet the PhD requirements for your institution
“PhD students and their supervisors often presume things without checking. One supervisor told his student that a PhD was about 300 pages long so he wrote 300 pages. Unfortunately the supervisor had meant double-spaced, and the student had written single-spaced. Getting rid of 40,000 extra words with two weeks to go is not recommended.” (Hannah Farrimond, lecturer in medical sociology, Exeter University)

2) Keep perspective
“Everyone wants their thesis to be amazing, their magnum opus. But your most important work will come later. Think of your PhD as an apprenticeship. Your peers are unlikely to read your thesis and judge you on it. They are more likely to read any papers (articles, chapters, books) that result from it.” (Dean D’Souza, PhD in cognitive neuroscience, Birkbeck, University of London)

3) Write the introduction last
“Writing the introduction and conclusion together will help to tie up the thesis together, so save it for the end.” (Ashish Jaiswal, PhD in business education, University of Oxford)

4) Use apps
“Trello is a project management tool (available as a smartphone app) which allows you to create ‘boards’ on which to pin all of your outstanding tasks, deadlines, and ideas. It allows you to make checklists too so you know that all of your important stuff is listed and to-hand, meaning you can focus on one thing at a time. It’s satisfying to move notes into the ‘done’ column too.” (Lucy Irving, PhD in psychology, Middlesex University)

5) Address the unanswered questions
“There will always be unanswered questions – don’t try to ignore or, even worse, obfuscate them. On the contrary, actively draw attention to them; identify them in your conclusion as areas for further investigation. Your PhD viva will go badly if you’ve attempted to disregard or evade the unresolved issues that your thesis has inevitably opened up.” (Michael Perfect, PhD in English literature, University of Cambridge)

6) Buy your own laser printer
“A basic monochrome laser printer that can print duplex (two-sided) can be bought online for less than £100, with off-brand replacement toners available for about £30 a pop. Repeatedly reprinting and editing draft thesis chapters has two very helpful functions. Firstly, it takes your work off the screen and onto paper, which is usually easier to proof. Secondly, it gives you a legitimate excuse to get away from your desk.” (James Brown, PhD in architectural education, Queen’s University Belfast)

7) Checking is important
“On days when your brain is too tired to write, check quotations, bibliography etc so you’re still making progress.” (Julia Wright, professor of English at Dalhousie University, Canada)

8) Get feedback on the whole thesis
“We often get feedback on individual chapters but plan to get feedback from your supervisor on the PhD as a whole to make sure it all hangs together nicely.” (Mel Rohse, PhD in peace studies, University of Bradford)

9) Make sure you know when it will end
“Sometimes supervisors use optimistic words such as ‘You are nearly there!’ Ask them to be specific. Are you three months away, or do you have six months’ worth of work? Or is it just a month’s load?” (Rifat Mahbub, PhD in women’s studies, University of York)

10) Prepare for the viva
“Don’t just focus on the thesis – the viva is very important too and examiners’ opinions can change following a successful viva. Remember that you are the expert in your specific field, not the examiners, and ask your supervisor to arrange a mock viva if practically possible.” (Christine Jones, head of school of Welsh and bilingual studies, University of Wales Trinity St David)

11) Develop your own style
“Take into account everything your supervisor has said, attend to their suggestions about revisions to your work but also be true to your own style of writing. What I found constructive was paying attention to the work of novelists I enjoy reading. It may seem that their style has nothing to do with your own field of research, but this does not matter. You can still absorb something of how they write and what makes it effective, compelling and believable.” (Sarah Skyrme, PhD in sociology, Newcastle University)

12) Remember that more is not always better
“A PhD thesis is not a race to the highest page count; don’t waste time padding.” (Francis Woodhouse, PhD in mathematical biology, University of Cambridge)

13) Get a buddy
“Find a colleague, your partner, a friend who is willing to support you. Share with them your milestones and goals, and agree to be accountable to them. This doesn’t mean they get to hassle or nag you, it just means someone else knows what you’re up to, and can help to check if your planning is realistic and achievable.” (Cassandra Steer, PhD in criminology, University of Amsterdam)

14) Don’t pursue perfectionism
“Remember that a PhD doesn’t have to be a masterpiece. Nothing more self-crippling than perfectionism.” (Nathan Waddell, lecturer in modernist literature, Nottingham University)

15) Look after yourself
“Go outside. Work outside if you can. Fresh air, trees and sunshine do wonders for what’s left of your sanity.” (Helen Coverdale, PhD in law, LSE)

• Do you have any tips to add? Share your advice in the comments below.

Join the higher education network for more comment, analysis and job opportunities, direct to your inbox. Follow us on Twitter @gdnhighered.

One thought on “Phd Dissertation How Many Words Is This

Leave a Reply

Your email address will not be published. Required fields are marked *