-1

As mentioned in the title, there are 200,000 plus course available in the website.. I have used the api using the tagging codes and dates but could only extract up to about 400 courses from the directory... Is there any way to obtain all the course reference number?

Below is my code

tagging_codes=[30011,30012,30013,30021,30022,30031,30032,30033,30041,30042,30043, 30051,30052,30053,30061,30062,30063,30071,30072,30073,30081,30082, 30083] full_list=[] section=['directory?pageSize=20&page='+str(num+1)+'&taggingCodes='+str(i)+'&courseSupportEndDate=20201223&retrieveType=FULL' for num in range(20) for i in tagging_codes] for i in section: res=requests.get('https://public-api.ssg-wsg.sg/courses/'+i,headers={'Authorization':'Bearer '+token_value}) soup=res.json() try: data=soup['data']['courses'] except: continue for j in range(len(data)): full_list.append(data_dict_maker(data,j)) # generate sleep sleep_duration = random.randint(2,6) print(sleep_duration, i, j) time.sleep(sleep_duration)
Improve this question

1 Answer 1

Reset to default
0

I checked the API response payload of the Retrieve Courses API and was able to retrieve more than 20,000 courses instead of the 400 courses you mentioned. MySkillsFuture’s courses search is also showing around 21k courses in total as well. Can share what was the input parameters used?

Improve this answer
Sign up to request clarification or add additional context in comments.

3 Comments

i have updated my original post with the code i used
Did you try “&taggingCodes=FULL” instead of the array of taggingCodes, as suggested in the Retrieve Courses API documents ? I tried and was able to retrieve all courses.
I saw from your code snippet that you are retrieving the token_value. Have you tried updating this? This Bearer Access token has an expiration of 3600 seconds only. Maybe, you can refer to this link developer.ssg-wsg.sg/webapp/guides/….

Start asking to get answers

Find the answer to your question by asking.

Ask question

Explore related questions

See similar questions with these tags.