UMAP2016EA

Analyzing MOOC Entries of Professionals on LinkedIn for User Modeling and Personalized MOOCRecommendations

[UMAP2016 submission by Guangyuan Piao and John G. Breslin]

About

This post provides supplemental material and information about the poster "Analyzing MOOC Entries of Professionals on LinkedIn for User Modeling and Personalized MOOC Recommendations: a first look". Available online: 

Poster:

UMAP2016EA - Analyzing MOOC Entries of Professionals on LinkedIn for User Modeling and Personalized MOOC Recommendations from GUANGYUAN PIAO

Dataset 

name	number of records	description
users.sql	5668	5668 learner profiles from LinkedIn who have been taken any Coursera MOOCs
coruseRecordsV1.sql	15744	course records extracted from user profiles
eduExperience.sql	11085	educational experience of learners
workExperience.sql	32801	work experience of learners
skills.sql	159291	skills of learners

Descriptive statistics: the dataset is about analyzed MOOC learner profiles from LinkedIn, which consists of 15,744 MOOC entries from 5,668 professionals. Each professional took 3 courses on average with the majority of learners (87%) having less than or equal to 5 MOOCs. Interestingly, the learner with the largest number of MOOCs had 114 of them. The distribution of genders and degrees of learners is as below:

If we assume that course entries in LinkedIn are courses that have been completed by users, the distribution of degrees are similar to the study [1] which provides the distribution of learners who completed their course.

Verified certifications: Instead of just taking MOOCs on Coursera and getting statements of accomplishment, learners can also purchase verified certifications for some courses that meet certain criteria. A verified certification provides proof that learners have completed their online courses. In such cases, varied certifications can also be added to LinkedIn parallels with their varied serial numbers. We found that around 26% of certifications in our collected profiles are verified while 74% of the certifications are unverified. 

Course tracks. We found that course tracks can be identified by exploring learning activities of users in the OSN. Formally, we can define a course track as a set of courses that were taken together more than n times where n is a threshold. The course relationships can be represented by weighted undirected networks like in the figure below. 

Nodes denote courses and the ties among courses denote the frequency of two courses taken together. In this context, a course track is a clique (or complete graph that has an edge joining each pair of nodes) within the course relationships network, with the weight of each tie in the clique is higher than the threshold n. Course tracks can be constructed based on the cliques within the course relationships network. As one might expect, the higher of the value n, the stronger the relationships a course track must hold with less number of cliques meeting the criteria. Indeed, 60 maximal cliques (a clique in maximal if it cannot be extended to a larger clique) can be found with a threshold of 10 while 16 maximal cliques can be found with a threshold of 20. 

We evaluated these tracks and found that two of the course tracks provided by Coursera can be identified in those cliques through this approach. A course track, called a specialization in Coursera, is a targeted sequence of courses from an institution taken together to earn a specialization certificate. The first course track from Coursera is a specialization of "Data Science" which consists of 9 courses from Johns Hopkins University, and the second one is a specialization of "Business Foundations" provided by the University of Pennsylvania. In practice, these ground truth course tracks can also be used for identifying the threshold n, which is the highest value that does not break the ground truth course tracks. In our case, 27 maximal cliques can be found including the two golden truth course tracks with the value of 13 for the threshold. Interestingly, when we look at the maximal clique that contains the "Data Science" course track (Figure 2), we found that "Machine Learning", "Introduction to Data Science" and "Computing for Data Analysis" are also being taken frequently with 9 courses in the "Data Science" course track in practice. This indicates that new course tracks can be constructed on top of existing tracks by exploring learning activities of users from the OSN.

[1]. T. Balch. MOOC student demographics. Retrieved Apr, 28:2013, 2013.

LinkedIn Recommendation

It's really good to see the practical recommendation case from industry via MOOCs (Introduction to Recommender Systems).

• Hybrid

Anmol Bhasin introduces the LinkedIn recsys overview and the system has various recsys for different tasks (job, people, article/newsfeed etc.) with hybrid strategies. The top-k recommendations from different recsys are mixed and optimized for a particular P (click rate, revenue, connection rate etc.) where constraints can denote such as certain portion of clicks to LinkedIn pages and others to external pages.

• Magic is not in a model, but features

Data is tremendous, but take into account the right features to models is more important. For instance, location is very important, but sometimes has been ignored by recsys. Students care less about locations while professionals don't.

• Use extensively crowdsourcing for learning something from different context data

• Different features and models for different populations (tuning regarding to the segments of the population)

• Evaluation offline an online
• Evaluation metrics may/may not correlate to the actual effects (interaction in the two settings is different, e.g., if show many times of an item online, then the user might click it - which cannot be captured in offline settings)
• A/B testing (sometimes called split testing) is comparing two versions of a web page to see which one performs better: if the evaluation metrics you were testing is promising, then do the A/B testing with segments of people by different settings of traffic (recruiters, talents).

Good practices of learning MOOCs

MOOCs - "On average 22 mins to answer the questions on wherever the students from can clearly find out common mistakes that students take (2000 much easier to determine than 2 in 200 offline classes)" [1-2]

Learning paradigm is changing: Lecture -> Mastery teaching -> Individual teaching (Lifelong needs for learning and adapt to the fast changing tech world)

Spend more time in interacting and creative improvement, learn by themselves with MOOCs

Several tips from a Facebook friend which are quite useful to me:

1. Check the introduction video
2. Do not register if you think you could not finish it
3. Learn it at the same time of the day as an offline course and get rid of any distraction during the course
4. Participate in learners' community
5. Do peer grading which actually help you to learn more by knowing the thoughts of other students
6. Tweet your registration (Is it improving success rate?? Interesting research question maybe...)
7. Restrict the quantity of MOOCs and improve the quality of your performance of MOOCs

Several take away from good MOOCs (let me continued and finished them):

1. Encouragement from the lecturer is really important in the start and end of the video lectures.
2. Make things clear and easy since there's no direct interaction of the lecture (and diversity of learners' levels).

My learnt MOOCs:

(O) Introduction to recommendation systems (Coursera)

(U) Probability : MITx: 6.041x Introduction to Probability - The Science of Uncertainty (edx)

(F) Statistics : Data to Insight (Future Learn)

(U) Social Network Analysis : Social Network Analysis (Coursera)

(F) Data Mining : Data Mining with Weka & More Data Mining with Weka

(F) Introduction to SAP HANA (OpenSAP)

(F) Introduction to SW development on SAP HANA (OpenSAP)

U: Unfinished
F: Finished

References:

[1]. http://www.ted.com/talks/daphne_koller_what_we_re_learning_from_online_education?language=en#t-1214445

[2]. http://www.ted.com/talks/salman_khan_let_s_use_video_to_reinvent_education