2

I'm trying to send an application after filling in a form available in a webpage using python. I've tried to mimic the process that I see in chrome dev tools but it seems I've gone somewhere wrong and that is the reason when I execute the following script I get this error:

{
"message":"415 Unsupported Media Type returned for /apply-app/rest/jobs/PIDFK026203F3VBQB79V77VIY-87592/submissions with message: ",
"key":"Exception_server_error",
"errorId":"d6b128bd-426d-4bee-8dbb-03e232829f5e"
}

It seems to me that I need to use the value of token and version in an automatic manner as they are different in every application but I don't find them in page source and stuff.

I've selected No as value for all the dropdowns (when there is any) within Additional Information.

Link to the application page

Link to the attachment that I've used thrice.

I've tried with:

import requests

main_link = "https://karriere.hsbc.de/stellenangebote/stellenboerse/apply?jobId=PIDFK026203F3VBQB79V77VIY-87592&langCode=de_DE"
post_link = "https://emea3.recruitmentplatform.com/apply-app/rest/jobs/PIDFK026203F3VBQB79V77VIY-87592/submissions"

payload = {
    "candidateIdentity":{"firstName":"syed","lastName":"mushfiq","email":"mthmt80@gmail.com"},
    "answeredDocuments":[{"documentType":"answeredForm","formId":"hsbc_bewerbungsprozess_pers_nliche_daten",
    "answers":[
    {"questionId":"form_of_address","type":"options","value":["form_of_address_m"]},
    {"questionId":"academic_title","type":"simple","value":" Dr.","questionIds":[]},
    {"questionId":"first_name","type":"simple","value":"syed","questionIds":[]},
    {"questionId":"last_name","type":"simple","value":"mushfiq","questionIds":[]},
    {"questionId":"e-mail_address","type":"simple","value":"mthmt80@gmail.com","questionIds":[]},
    {"questionId":"phone__mobile_","type":"phone","countryCode":"+880","isoCountryCode":"BD","subscriberNumber":"1790128884"}]},
    {"documentType":"answeredForm","formId":"hsbc_bewerbungsprozess_standard_fragebogen","answers":[{"questionId":"custom_question_450","type":"options","value":["custom_question_450_ja"]},
    {"questionId":"custom_question_451","type":"options","value":["custom_question_451_nein"]},
    {"questionId":"custom_question_452","type":"options","value":["custom_question_452_unter_keine_der_zuvor_genannten"]},
    {"questionId":"custom_question_580","type":"options","value":["custom_question_580_nein_978"]},
    {"questionId":"custom_question_637","type":"options","value":["custom_question_637_nein"]},
    {"questionId":"custom_question_579","type":"options","value":["custom_question_579_nein"]},
    {"questionId":"custom_question_583","type":"options","value":["custom_question_583_hsbc_deutschland_karriereseite"]}]},

    #============The following three lines are supposed to help upload three files============

    {"documentType":"attachment","attachmentId":"cover_letter","token":"2d178469-cdb5-4d65-9f67-1e7637896953","filename": open("demo.pdf","rb")},
    {"documentType":"attachment","attachmentId":"attached_resume","token":"81a5a661-66bb-4918-a35c-ec260ffb7d02","filename": open("demo.pdf","rb")},
    {"documentType":"attachment","attachmentId":"otherattachment","token":"4c3f7500-b072-48d4-83cf-0af1399bc8ba","filename": open("demo.pdf","rb")}],

    #============The version's value should not be hardcoded=========================

    "version":"V2:3:14dfac80702d099625d0274121b0dba68ac0fd96:861836b7d86adae8cc1ce69198b69b8ca59e2ed5","lastModifiedDate":1562056029000,"answeredDataPrivacyConsents":[{"identifier":"urn:lms:ta:tlk:data-privacy-consent:mtu531:101","consentProvided":True},
    {"identifier":"urn:lms:ta:tlk:data-privacy-consent:mtu531:102","consentProvided":True}],
    "metaInformation":{"applicationFormUrl":"https://karriere.hsbc.de/stellenangebote/stellenboerse/apply?jobId=PIDFK026203F3VBQB79V77VIY-87592&langCode=de_DE","jobsToLink":[]}
}

def send_application(s,link):   
    res = s.post(link,data=payload)
    print(res.text)

if __name__ == '__main__':
    with requests.Session() as s:
        send_application(s,post_link)

How can I send the application in the right way?

PS I can send the application manually multiple times using the same documents to the same email.

robots.txt
  • 96
  • 2
  • 10
  • 36
  • Are you open for selenium based approach? it wont be as fast as requests, but will save you dev time. – Chandan Gm Apr 18 '20 at 11:51
  • Thanks for your interest @Chandan Gm. I would go for selenium as a last resort, If I did not get the requests based solution already. It seems the solution provided by bmcculley is working in the right way. – robots.txt Apr 18 '20 at 12:13

1 Answers1

4

The best way to go about something like this is to open the page in a browser and view the network tab in the developer tools. From there as you're filling out the form you'll be able to see that each time you attach a document it sends an ajax request and receives the token in a json response. With those tokens you can build the final payload which should be submitted in json format.

Here's some example code that's working:

import requests

headers = {
    'Host': 'emea3.recruitmentplatform.com',
    'User-Agent': 'Mozilla/5.0 (Windows NT 10.0; Win64; x64) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/64.0.3282.140 Safari/537.36 Edge/17.17134',
    'Accept': 'application/json, text/javascript, */*; q=0.01',
    'Accept-Language': 'en-US,en;q=0.5',
    'Accept-Encoding': 'gzip, deflate',
    'apply-config-key': 'AAACEwAA-55cd88d4-c9fd-41ce-95a4-f238402b898f',
    'Origin': 'https://karriere.hsbc.de',
    'DNT': '1',
    'Connection': 'close',
    'Referer': 'https://karriere.hsbc.de/',
    'Cookie': 'lumesse_language=de_DE'
}

main_link = "https://karriere.hsbc.de/stellenangebote/stellenboerse/apply?jobId=PIDFK026203F3VBQB79V77VIY-87592&langCode=de_DE"
post_link = "https://emea3.recruitmentplatform.com/apply-app/rest/jobs/PIDFK026203F3VBQB79V77VIY-87592/submissions"
ajax_link = "https://emea3.recruitmentplatform.com/apply-app/rest/jobs/PIDFK026203F3VBQB79V77VIY-87592/attachments"

def build_payload(cover_letter_token, attached_resume_token, otherattachment_token):
    return {
      "candidateIdentity": {
        "firstName": "Syed",
        "lastName": "Mushfiq",
        "email": "mthmt80@gmail.com"
      },
      "answeredDocuments": [
        {
          "documentType": "answeredForm",
          "formId": "hsbc_bewerbungsprozess_pers_nliche_daten",
          "answers": [
            {
              "questionId": "form_of_address",
              "type": "options",
              "value": [
                "form_of_address_m"
              ]
            },
            {
              "questionId": "academic_title",
              "type": "simple",
              "value": "prof",
              "questionIds": []
            },
            {
              "questionId": "first_name",
              "type": "simple",
              "value": "Syed",
              "questionIds": []
            },
            {
              "questionId": "last_name",
              "type": "simple",
              "value": "Mushfiq",
              "questionIds": []
            },
            {
              "questionId": "e-mail_address",
              "type": "simple",
              "value": "mthmt80@gmail.com",
              "questionIds": []
            },
            {
              "questionId": "phone__mobile_",
              "type": "phone",
              "countryCode": "+49",
              "isoCountryCode": "DE",
              "subscriberNumber": "30 33850062"
            }
          ]
        },
        {
          "documentType": "answeredForm",
          "formId": "hsbc_bewerbungsprozess_standard_fragebogen",
          "answers": [
            {
              "questionId": "custom_question_450",
              "type": "options",
              "value": [
                "custom_question_450_ja"
              ]
            },
            {
              "questionId": "custom_question_451",
              "type": "options",
              "value": [
                "custom_question_451_nein"
              ]
            },
            {
              "questionId": "custom_question_452",
              "type": "options",
              "value": [
                "custom_question_452_unter_keine_der_zuvor_genannten"
              ]
            },
            {
              "questionId": "custom_question_580",
              "type": "options",
              "value": [
                "custom_question_580_ja"
              ]
            },
            {
              "questionId": "custom_question_637",
              "type": "options",
              "value": [
                "custom_question_637_nein"
              ]
            },
            {
              "questionId": "custom_question_579",
              "type": "options",
              "value": [
                "custom_question_579_nein"
              ]
            },
            {
              "questionId": "custom_question_583",
              "type": "options",
              "value": [
                "custom_question_583_linkedin"
              ]
            }
          ]
        },
        {
          "documentType": "attachment",
          "attachmentId": "cover_letter",
          "token": cover_letter_token,
          "filename": "demo.pdf"
        },
        {
          "documentType": "attachment",
          "attachmentId": "attached_resume",
          "token": attached_resume_token,
          "filename": "demo.pdf"
        },
        {
          "documentType": "attachment",
          "attachmentId": "otherattachment",
          "token": otherattachment_token,
          "filename": "demo.pdf"
        }
      ],
      "version": "V2:3:14dfac80702d099625d0274121b0dba68ac0fd96:861836b7d86adae8cc1ce69198b69b8ca59e2ed5",
      "lastModifiedDate": "1562056029000",
      "answeredDataPrivacyConsents": [
        {
          "identifier": "urn:lms:ta:tlk:data-privacy-consent:mtu531:101",
          "consentProvided": "true"
        },
        {
          "identifier": "urn:lms:ta:tlk:data-privacy-consent:mtu531:102",
          "consentProvided": "true"
        }
      ],
      "metaInformation": {
        "applicationFormUrl": "https://karriere.hsbc.de/stellenangebote/stellenboerse/apply?jobId=PIDFK026203F3VBQB79V77VIY-87592&langCode=de_DE",
        "jobsToLink": []
      }
    }

def submit_attachment(s, link, f):
    d = open(f, 'rb').read()
    r = s.post(link, files={'file':('demo.pdf', d),'applicationProcessVersion':(None, 'V2:3:14dfac80702d099625d0274121b0dba68ac0fd96:861836b7d86adae8cc1ce69198b69b8ca59e2ed5')})
    r_data = r.json()
    return r_data.get('token')

def send_application(s,link,p):   
    res = s.post(link, json=p)
    return res

if __name__ == '__main__':
    attachment_list = ["cover_letter_token", "attached_resume_token", "otherattachment_token"]
    token_dict = {}

    with requests.Session() as s:
        s.headers.update(headers)

        for at in attachment_list:
            rt = submit_attachment(s, ajax_link, "demo.pdf")
            token_dict[at] = rt

        payload = build_payload(token_dict['cover_letter_token'], token_dict['attached_resume_token'], token_dict['otherattachment_token'])

        rd = send_application(s, post_link, payload)
        print(rd.text)
        print(rd.status_code)
bmcculley
  • 2,048
  • 1
  • 14
  • 17
  • It appears to be working in the right way @bmcculley. It would be great if you bring about a small change within your solution so that I can use three different pdf files, as in `first_demo.pdf`,`second_demo.pdf` and `third_demo.pdf` instead of just `demo.pdf` in all three cases. Thanks. – robots.txt Apr 18 '20 at 17:12
  • Hi @bmcculley, I've modified like [this](https://filebin.net/nke6gabpjgj7xa2p) to make use of all three pdf files and I find it working. Did I not do the modification in the right way? Thanks. – robots.txt Apr 19 '20 at 05:12
  • It would be great if you give [this post](https://stackoverflow.com/questions/61280657/unable-to-scrape-the-name-from-the-inner-page-of-each-result-using-requests) a go in case there is any solution to offer @bmcculley. Thanks. – asmitu Apr 21 '20 at 05:40