# Plot Network Data in R with iGraph

I recently had a conversation on Twitter about a plot I made a while back. Recall, the plot showed my Twitter network, my friends and my friend’s friends.

And here’s the R code:

#### Load R libraries
library("iGraph")

#### Convert to graph object
gr <- graph.data.frame(r,directed=TRUE)

#### gr
# Describe graph
summary(gr)
ecount(gr) # Edge count
vcount(gr) # Node count
diameter(gr) # Network diameter
farthest.nodes(gr) # Nodes furthest apart
V(gr)$indegree = degree(gr,mode="in") # Calculate indegree #### Plot graph E(gr)$color = "gray"
E(gr)$width = .5 E(gr)$arrow.width = .25
V(gr)$label.color = "black" V(gr)$color = "dodgerblue"
V(gr)$size = 4 set.seed(40134541) l <- layout.fruchterman.reingold(gr) pdf("network_friends_plot.pdf") plot(gr,layout=l,rescale=TRUE,axes=FALSE,ylim=c(-1,1),asp=0,vertex.label=NA) dev.off() Advertisement # Create a dictionary of authors and author attributes and values for a journal article using the Scopus API and Python As an exercise to brush up my Python skills, I decided to tinker around with the Scopus API. Scopus is an online database maintained by Elsevier that records and provides access to information about peer reviewed publications. Not only does Scopus let users search for journal articles based on key words and various other criteria, but the web services also allows users to explore these articles as networks of articles, authors, institutions, and so forth. If you’re interested in risk factors that lead to scholarly publications, publication citations, or impact factors, this is a place to start. The following code yields a dictionary of author information by requesting content through the abstract retrieval API. This request is made using the Python package requests and parsed using the package BeautifulSoup. Enjoy! #### Import python packages import requests from bs4 import BeautifulSoup #### Set API key my_api_key = 'YoUr_ApI_kEy' #### Abstract retrieval API # API documentation at http://api.elsevier.com/documentation/AbstractRetrievalAPI.wadl # Get article info using unique article ID eid = '2-s2.0-84899659621' url = 'http://api.elsevier.com/content/abstract/eid/' + eid header = {'Accept' : 'application/xml', 'X-ELS-APIKey' : my_api_key} resp = requests.get(url, headers=header) print 'API Response code:', resp.status_code # resp.status_code != 200 i.e. API response error # Write response to file #with open(eid, 'w') as f: # f.write(resp.text.encode('utf-8')) soup = BeautifulSoup(resp.content.decode('utf-8','ignore'), 'lxml') soup_author_groups = soup.find_all('author-group') print 'Number author groups:', len(soup_author_groups) author_dict = {} # Traverse author groups for i in soup_author_groups: # Traverse authors within author groups for j in i.find_all('author'): author_dict.update({j.attrs['auid']:j.attrs}) # Return dictionary of attributes j.contents.pop(-1) # Pop dicitonary of attributes # Traverse author contents within author for k in j.contents: author_dict[j.attrs['auid']].update({k.name : k.contents[0]}) print author_list # Download Twitter Data using JSON in R Here we consider the task of downloading Twitter data using the R software package RJSONIO. Before we can download Twitter data, we’ll need to prove to Twitter that we are in fact authorized to do so. I refer the interested reader to the post Twitter OAuth FAQ for instructions on how to setup an application with dev.twitter.com. Once we’ve setup an application with Twitter we can write some R code to communicate with Twitter about our application and get the data we want. Code from the post Authorize a Twitter Data request in R, specifically the keyValues() function, will be used in this post to handle our authentication needs when requesting data from Twitter. ## Install R packages install.packages('bitops') install.packages('digest') install.packages('RCurl') install.packages('ROAuth') install.packages('RJSONIO') ## Load R packages library('bitops') library('digest') library('RCurl') library('ROAuth') library('RJSONIO') library('plyr') ## Set decimal precision options(digits=22) ## OAuth application values oauth <- data.frame(consumerKey='YoUrCoNsUmErKeY',consumerSecret='YoUrCoNsUmErSeCrEt',accessToken='YoUrAcCeSsToKeN',accessTokenSecret='YoUrAcCeSsToKeNsEcReT') keyValues <- function(httpmethod,baseurl,par1a,par1b){ # Generate a random string of letters and numbers string <- paste(sample(c(letters[1:26],0:9),size=32,replace=T),collapse='') # Generate random string of alphanumeric characters string2 <- base64(string,encode=TRUE,mode='character') # Convert string to base64 nonce <- gsub('[^a-zA-Z0-9]','',string2,perl=TRUE) # Remove non-alphanumeric characters # Get the current GMT system time in seconds timestamp <- as.character(floor(as.numeric(as.POSIXct(Sys.time(),tz='GMT')))) # Percent encode parameters 1 #par1 <- '&resources=statuses' par2a <- gsub(',','%2C',par1a,perl=TRUE) # Percent encode par par2b <- gsub(',','%2C',par1b,perl=TRUE) # Percent encode par # Percent ecode parameters 2 # Order the key/value pairs by the first letter of each key ps <- paste(par2a,'oauth_consumer_key=',oauth$consumerKey,'&oauth_nonce=',nonce[1],'&oauth_signature_method=HMAC-SHA1&oauth_timestamp=',timestamp,'&oauth_token=',oauth$accessToken,'&oauth_version=1.0',par2b,sep='') ps2 <- gsub('%','%25',ps,perl=TRUE) ps3 <- gsub('&','%26',ps2,perl=TRUE) ps4 <- gsub('=','%3D',ps3,perl=TRUE) # Percent encode parameters 3 url1 <- baseurl url2 <- gsub(':','%3A',url1,perl=TRUE) url3 <- gsub('/','%2F',url2,perl=TRUE) # Create signature base string signBaseString <- paste(httpmethod,'&',url3,'&',ps4,sep='') # Create signing key signKey <- paste(oauth$consumerSecret,'&',oauth$accessTokenSecret,sep='') # oauth_signature osign <- hmac(key=signKey,object=signBaseString,algo='sha1',serialize=FALSE,raw=TRUE) osign641 <- base64(osign,encode=TRUE,mode='character') osign642 <- gsub('/','%2F',osign641,perl=TRUE) osign643 <- gsub('=','%3D',osign642,perl=TRUE) osign644 <- gsub('[+]','%2B',osign643,perl=TRUE) return(data.frame(hm=httpmethod,bu=baseurl,p=paste(par1a,par1b,sep=''),nonce=nonce[1],timestamp=timestamp,osign=osign644[1])) }  Next, we need to figure out what kind of Twitter data we want to download. The Twitter REST API v1.1 Resources site provides a useful outline of what kind of data we can get from Twitter. Just read what is written under the Description sections. As an example, let’s download some user tweets. To do this, we find and consult the specific Resource on the REST API v1.1 page that corresponds with the action we want, here GET statuses/user_timeline. The resource page lists and describes the download options available to the task of getting tweets from a specific user, the thing we want to do, so it’s worth it to the reader to check it out. Here we download the 100 most recent tweets (and re-tweets) made by the user ‘Reuters’. ## Download user tweets # Limited to latest 200 tweets # Specify user name user <- 'Reuters' kv <- keyValues(httpmethod='GET',baseurl='https://api.twitter.com/1.1/statuses/user_timeline.json',par1a='count=100&include_rts=1&',par1b=paste('&screen_name=',user,sep='')) theData1 <- fromJSON(getURL(paste(kv$bu,'?','oauth_consumer_key=',oauth$consumerKey,'&oauth_nonce=',kv$nonce,'&oauth_signature=',kv$osign,'&oauth_signature_method=HMAC-SHA1&oauth_timestamp=',kv$timestamp,'&oauth_token=',oauth$accessToken,'&oauth_version=1.0','&',kv$p,sep='')))


At this point in the post you should have the 100 most recent tweets made by the user ‘Reuters’ as well as values on several variables recorded by Twitter on each tweet. These are stored in a list data structure. You are now free to do list things to these data to explore what it is you have.

For instance, let’s see the tweets.

theData2 <- unlist(theData1)
names(theData2)
tweets <- theData2[names(theData2)=='text']