Re: Is there any 'xoauth2' login wrapper support for v5 of current getmail for ms o365 in the works?

On Sun, Jan 09, 2022 at 03:50:37PM -0500, Pieter Jacques wrote:
> Donald,
> 
> My situation seems identical to yours:  a university email system that uses
> Microsoft Office365 with Duo 2-factor authentication, and which now requires
> xoauth2.  I got this working with Stefan Krah's getmail-gmail-xoauth-tokens
> as follows.
> 
> First of all, for this to work your University must allow xoauth2 access
> from Thunderbird.  Probably most universities do allow Thunderbird, since
> it's very popular.  An interesting point here is that the xoauth2 client ID
> and client secret for Thunderbird are not exactly "secret". Since
> Thunderbird is open source, anyone can download the Thunderbird source code
> and extract the ID and secret.  (Try googling "thunderbird
> oauth2providers.jsm")
> 
> Follow the instructions on the "getmail-5.6 XOAUTH2 configuration for gmail"
> web page, with the following changes:
> 
> 1) Instead of the suggested gmail.json file, create a "msoft.json" file,
> using this template:
> 
> {"scope": "offline_access https://urldefense.com/v3/__https://outlook.office365.com/IMAP.AccessAsUser.All__;!!LIr3w8kk_Xxm!7hHKrLqEOnWwU-bPoh1rdqMwSE_-COoKeA9p9F69S58C3_ea0Pgyi4jyzsVSlzA$ ",
>  "response_type": "code",
>  "client_id": "INSERT THUNDERBIRD CLIENT ID",
>  "client_secret": "INSERT THUNDERBIRD CLIENT SECRET",
>  "redirect_uri": "https://urldefense.com/v3/__http://localhost__;!!LIr3w8kk_Xxm!7hHKrLqEOnWwU-bPoh1rdqMwSE_-COoKeA9p9F69S58C3_ea0Pgyi4jyD2ajSQ4$ ",
>  "token_uri": "https://urldefense.com/v3/__https://login.microsoftonline.com/common/oauth2/v2.0/token__;!!LIr3w8kk_Xxm!7hHKrLqEOnWwU-bPoh1rdqMwSE_-COoKeA9p9F69S58C3_ea0Pgyi4jyYYq2_CQ$ ",
>  "auth_uri": "https://urldefense.com/v3/__https://login.microsoftonline.com/common/oauth2/v2.0/authorize__;!!LIr3w8kk_Xxm!7hHKrLqEOnWwU-bPoh1rdqMwSE_-COoKeA9p9F69S58C3_ea0Pgyi4jytoley18$ ",
>  "user": "[email protected]"}
> 
> 2) Next do "getmail-gmail-xoauth-tokens --init PATH-TO-MSOFT.JSON
> 
> 3) This will output a URL.  Mouse-copy that URL into a browser and open the
> URL.  This should take to your university's login procedure, complete with
> Duo.  Enter login information as required.
> 
> 4) Once you complete entering the login information your browser will give
> an "Unable to connect" error.  This is expected.  DO NOT CLOSE THE BROWSER
> WINDOW.  Instead, look in the URL box, which should look something like
>    "localhost/?code=A-LONG-STRING&session_state=ANOTHER-STRING"
> 
> 5) Mouse copy the "A-LONG-STRING" part of that, and paste it into the "Enter
> verification code" prompt from getmail-gmail-xoauth-tokens.
> 
> This should result in an updated msoft.json file that will allow getmail to
> fetch email from Microsoft Office365 with xoauth2 authentication.
> 
> - Pieter
> 
Hello Pieter et. al.,

I think I followed your excellent instructions but I must have gotten something wrong, and I hope someone can help me spot the problem.  

Here is my msoft.json file:

{"scope": "offline_access https://outlook.office365.com/IMAP.AccessAsUser.All",
  "response_type": "code",
  "client_id": "08162f7c-0fd2-4200-a84a-f25a4db0b584",
  "client_secret": "TxRBilcHdC6WGBee]fs?QR:SJ8nI[g82",
  "redirect_uri": "http://localhost",
  "token_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/token",
  "auth_uri": "https://login.microsoftonline.com/common/oauth2/v2.0/authorize",
  "user": "[email protected]"}

I may have made some errors.  In addition to the xoauth and 2 factor auth. another thing this university does to make life difficult is to subscribe to Proofpoint, so every URL in every message is encased in a lot of gobbledy gook, which has to be eliminated so as to get down to the real URL.  I hope that the proofpoint stuff is eliminated in the copy of this that goes out to the list, but I can't really tell because it's there when I view it in my email.

When I do step 2 and run the script it does output a URL.

dmacdoug@medmedia:~/.getmail$ getmail-gmail-xoauth-tokens --init /home/dmacdoug/.getmail/msoft.json 

Visit this url to obtain a verification code:

    https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=08162f7c-0fd2-4200-a84a-f25a4db0b584&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=offline_access%20https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All



File msoft.json saved
dmacdoug@medmedia:~/.getmail$ getmail-gmail-xoauth-tokens --init /home/dmacdoug/.getmail/msoft.json 
Visit this url to obtain a verification code:

    https://login.microsoftonline.com/common/oauth2/v2.0/authorize?client_id=08162f7c-0fd2-4200-a84a-f25a4db0b584&redirect_uri=http%3A%2F%2Flocalhost&response_type=code&scope=offline_access%20https%3A%2F%2Foutlook.office365.com%2FIMAP.AccessAsUser.All

When I put that URL into a browser it takes me to my own webserver which pops up a login box which says:

http://localhost is requesting your username and password. The site says: “Authorized User”

Whether I give a correct or incorrect username and password and click submit or cancel I them am able to find "A_LONG_STRING" and paste it into the "Enter verification code:" prompt. when I hit 'Enter' the following prints out:

Traceback (most recent call last):
  File "/usr/bin/getmail-gmail-xoauth-tokens", line 113, in <module>
    response = auth.init_tokens(code)
  File "/usr/bin/getmail-gmail-xoauth-tokens", line 82, in init_tokens
    self.update_config(d)
  File "/usr/bin/getmail-gmail-xoauth-tokens", line 65, in update_config
    self.data['access_token'] = d['access_token']
KeyError: 'access_token'

At some point in some earlier iteration of this process several days ago I did at some point see an official looking microsoft page with a place to put in a name or password or something but it's dim in my mind now and I can't figure out how to replicate it and it definitely was not my university's login procedure.

If you see where I went wrong, I'd appreciate being steered right.

Thank you,
Don MacDougall