Win a copy of Succeeding with AI this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Paul Clapham
  • Ron McLeod
  • Liutauras Vilda
  • Junilu Lacar
Sheriffs:
  • Tim Cooke
  • Jeanne Boyarsky
  • Knute Snortum
Saloon Keepers:
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Carey Brown
  • Piet Souris
Bartenders:
  • salvin francis
  • fred rosenberger
  • Frits Walraven

OAuth in email app?

 
Ranch Hand
Posts: 101
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm trying to build Email messaging app in Spring Boot with Android app as client. Since for every Account it is required to have mail server configuration parameters, like so:



I still have to "Allow less secure app access" from my Google Account. Now, is there any way to get around this? When some User try to send his email, that he doesn't need to enable this button. And is it true that if OAuth is implemented during login, that my app will be considered "secure enough" so the User doesn't have to click that "Enable" button?
 
Saloon Keeper
Posts: 22011
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm forgetful of the nuances of OAuth, but I don't believe that it applies to email - just websites.

However, on an Android app, you probably shouldn't be sending email directly, but rather contacting the email/gmail intent. Which already has the requisite settings and credentials.
 
Ranch Foreman
Posts: 103
7
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
From as far as I know and understand OAuth there is a catch between developing something only for yourself and something you distribute to others: The client secret.
The OAuth authorization flow works like this: You, as the developer, have a client-id and a client-secret. To authenticate yourself against the OAuth endpoint you first put together an authorization URL to get an authorization token. In the second step to get the access and refresh token you build the token URL with the previous authorization token and your client-secret.
But: How does a customer not knowing your client-secret does this? My examples for several different platforms ended in something like this: On one of my public accessible servers I had set up what I call a relay. A customer ask my server to generate a new authorization URL - easy enough and as the client-id is public this can be done at the client. I do it on the server to have control over the scopes (it depends on what to user wants to do as different actions require different scopes). The user then just visit this URL with the browser and either accepts or denies the request. Depending on the platform used the users browser either shows some token to be copied (google does that) or is redirected to a callback page hosted on my server. From there one could implement the authorization in several differnt ways. The redirect has the unique authorization token. So, the service server sends this secured to the client and hence the site to where this redirect points sould also be secured (if possible only local). From there you can either read the token within your application or have the user to copy it manually (all following the OAuth2 RFC - which by itself is not so helpful).
After the user got the unique authorization token it is send to my server for further processing. To secure this against attacks it's recommend to use one-time-nounces. When the authorization token is received on my server then it gets the authorization and refresh token from the service and relays them back to the client without storing it on the server itself. It's possible to store them on your server and to have all data flown through your server, but this only makes sense within a company network where all data are have to go through the master proxy.
As the client then only gets the access token (which is only valid for a limited time) and the refresh token the client now can access GMail.
When the access token runs out google replies with an error. As from different sources I read it's up to you as the dev to take into account to either wait until the error is replied or refresh the access token before the timeout, but the RFC states that a client has to be able to handle errors produced by the end point. As available OAuth libraries expect to also having a client-secret (googles one does and is annoying to deal with) I quit this problem on this point. But I also came up with this one:
When the client gets the error about the access token timeout it requests a new access token via sending a request along with the refresh token to my server to handle it. This works, but it would take a lot of effort to re-write googles oauth lib to instead of trying a refresh itself without the client-secret to relay it via your server. I don't know why there're no libraries out there to handle this or so few information, as OAuth is widely used. From the point I just said to myself "screw this" I guess any service using OAuth has to handle this with something homebrew - as at leas I wasn't able to find a standard way to such relay action without build it myself.
Even worse: OAuth and Google don't really have their own support but rather redirect one to StackOverflow - and I had two or three topics open for very long time without any reply.
If you want to built this all your own I recommend to have at least two independed security audits have a look before publishing it - otherwise you somehow will end up in some security issues either make this all not work or leak secure information.
 
Saloon Keeper
Posts: 11923
253
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Bob Winter wrote:The OAuth authorization flow works like this


There is no "OAuth autorization flow". There are several authorization grant types that are part of the OAuth 2.0 spec. Which one to use depends on the type of your application and how it accesses the resource service. What you described is the Authorization Code grant type, which is NOT appropriate for client-side applications without using PKCE. PKCE means having the client generate a challenge on the fly, instead of using a hard-coded client secret.

In the second step to get the access and refresh token you build the token URL with the previous authorization token and your client-secret.
But: How does a customer not knowing your client-secret does this?


They don't. Your application backend on the server-side exchanges the authorization code for an access token using a client secret, NOT the client-side application. If your application is a single page application or mobile application that accesses the resource service directly instead of through your application backend, you MUST use PKCE. If you can't use PKCE (because the authorization service doesn't support it), then you must not exchange the authorization code for an access token from the client-side.

The redirect has the unique authorization token. So, the service server sends this secured to the client


No. EITHER the client parses the authorization code from the redirect URL and then exchanges it for an access token using PKCE, OR it lets the browser handle the redirect in which case the server-side will handle the exchange.

It's possible to store them on your server and to have all data flown through your server, but this only makes sense within a company network where all data are have to go through the master proxy.


I don't agree with this premise. The majority of applications access external services through the application backend. As a matter of fact, this is the only way to write a web application if you don't want to run any code on the client, such as in static web pages that don't use JavaScript.

When the client gets the error about the access token timeout it requests a new access token via sending a request along with the refresh token to my server to handle it. This works, but it would take a lot of effort to re-write googles oauth lib to instead of trying a refresh itself without the client-secret to relay it via your server.


I don't really see why you would need to rewrite anything. Either Google provides a library that you can use on the client-side that uses PKCE, or it provides a library that you can use on the server-side that uses a client secret, or both.

I don't know why there're no libraries out there to handle this or so few information, as OAuth is widely used. From the point I just said to myself "screw this"


In the majority of cases OAuth is used from the server-side, in which case you use Authorization Code grant type with client secret. For applications that want to access the resource service directly, people have historically used the Implicit Flow grant type, and still often do so because that's all they know. Implicit Flow is deprecated now because it's insecure. Instead, you need to use Authorization Code with PKCE. It's possible that PKCE is not widely supported because people tend to go "screw this" and stick with what they know.

[edit]
Bob, the last sentence of my post might seem like a dig at you. I assure you it wasn't meant this way. I was referring to people who write general purpose OAuth libraries without using updated recommended practices.
[/edit]
 
Stephan van Hulst
Saloon Keeper
Posts: 11923
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Mike Gosling wrote:And is it true that if OAuth is implemented during login, that my app will be considered "secure enough" so the User doesn't have to click that "Enable" button?


Yes, using OAuth should allow access to your mail server without having to "Allow less secure app access".
 
Saloon Keeper
Posts: 6391
158
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

However, on an Android app, you probably shouldn't be sending email directly, but rather contacting the email/gmail intent. Which already has the requisite settings and credentials.


+1. If for some reason no user interaction is desired, have the app contact a REST service on your server with the relevant details, and then the server can send out mails.
 
Mike Gosling
Ranch Hand
Posts: 101
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Thank you all for taking your time to post your answers.

Bob Winter wrote:From as far as I know and understand ...


This all is very vague for me since I'm beginner in Spring Boot and building web-apps in general. I'll try to use this, along with Stephan van Hulst's comments, in the future.

Tim Holloway wrote:However, on an Android app, you probably shouldn't be sending email directly, but rather contacting the email/gmail intent.


Since this is a sophomore student project, we cannot just contact gmail intent, tho, it would be much easier xD

Stephan van Hulst wrote:Yes, using OAuth should allow access to your mail server without having to "Allow less secure app access".


As I said that I'm new to building back-end web apps, can you maybe recommend some resource (tutorial, blog etc) on how to actually implement OAuth? By your comments, you seem to have an in-depth knowledge


Tim Moores wrote:If for some reason no user interaction is desired, have the app contact a REST service on your server with the relevant details, and then the server can send out mails.


You are correct Tim! We are supposed to send http request (using retorfit2) from mobile app to our back-end and then JavaMail library (along with appropriate mail server configuration parameters) will actually send email.


I do have one question if you guys can help I would be very grateful. Question is a bit off topic. Okey, I managed to send an email for someone who is using gmail, but what if a User have Account for lets say yahoo or hotmail or university mail. How to set (programmatically) mail server configuration parameters for each of this mail service, how do they differ? For Google, I found necessary https://support.google.com/mail/answer/7126229?visit_id=637242948460584640-1600276272&hl=en&rd=1. Bellow is part of our UML, which we need to follow up.


UML.png
[Thumbnail for UML.png]
 
Tim Moores
Saloon Keeper
Posts: 6391
158
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

I managed to send an email for someone who is using gmail, but what if a User have Account for lets say yahoo or hotmail or university mail. How to set (programmatically) mail server configuration parameters for each of this mail service, how do they differ?


There's the rub, if you want to use a solution like what you proposed. Using either an Intent, or a server-based solution where you can hand the email to the mail server, would sidestep that issue. Otherwise you'd have to have the user enter all the details you need in the app, like what you have to do when you set up a non-Gmail account in your Android email app.
 
Tim Holloway
Saloon Keeper
Posts: 22011
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Mik Gosling wrote:I managed to send an email for someone who is using gmail, but what if a User have Account for lets say yahoo or hotmail or university mail. How to set (programmatically) mail server configuration parameters for each of this mail service, how do they differ?



You basically just have to set all of the properties with values appropriate to the mailserver that you will be sending through. Same as with gmail, but each of those services has their own host address, port, security prototocols and connection credentials.

Incidentally:

https://javaee.github.io/javamail/OAuth2
 
Bob Winter
Ranch Foreman
Posts: 103
7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst - you don't have to apologize. As said: What I wrote is based upon how far I had to deal with OAuth yet and about how far I understand it. If this is only a part of OAuth which may not fit the question of OP and may contain wrong information please feel free to point them out and correct them if you have better knowledge of OAuth.
Just let me ask: You used the short "PKCE" - what is it in full words? I'm not familiar with these four letters.

Maybe let me explain my mess of thrown together stuff in the middle of the night:

For now I only dealt with OAuth in the way that I had to implement it to use it in my own personal codes. As far as I was able to find the one useful documentation comes from Google, the other services I tried only refered to "we use OAuth" - like "come up with your own way to implement it". Google also provides a OAuth lib (which, as far as I tested, seem to be optimized to be used with Googles services, as I wasn't able to use it with other services).
I used it this way: As I wanted to access a resource on google I generated an access on googles developer console - which in turn gives me a client-id and a client-secret. Following googles own documentation to use their lib with their services in java I had to create an instance of a class called AuthorizationCodeFlow. This class took four paramters: the client id, the client secret, and two urls (one for the autorization and one for the token). The doc then further explained this: request an authorization url and let the user do the stuff on googles site. As for the reply there're three different ways: a code that is displayed and has to be copied by hand by the user, then a way what google calls "programm extraction" and the third was using some callback url on a server. As I wasn't able to implement the latter two I chose the first option: let google display some code the user (in this case me) then had to copy back in the application. After this code was copied in then this AuthorizationCodeFlow was to be called with this code and got back the token, wich then could be used with the api client.
All this works well as far as I as the dev have the credentials - but I don'T know how to write it so that lets say a friend of mine could use my stuff without me have to give him the google credentials or to have to do all the processing on my server. But I guess that's just me yet not understood OAuth fully. I'm willing to learn but wasn't unable to find any other useful doc instead of googles. And theirs only got me so far to the mess I know now. If you have better doc sources I'd like to know them.
 
Tim Moores
Saloon Keeper
Posts: 6391
158
Android Mac OS X Firefox Browser VI Editor Tomcat Server Safari
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
https://oauth.net/2/pkce/
 
Bob Winter
Ranch Foreman
Posts: 103
7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, the oauth.net site is a bit like the BouncyCastle doc: Full of nothing - or: it has a lot of writing without saying anything. At least for me not helpful.
 
Tim Holloway
Saloon Keeper
Posts: 22011
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry about that. That's not the page I was looking at when I posted the link.

You might check the actual RFC. It's short and has a diagram.

I'm still unclear myself if OAUTH in email is supposed to actually grant access to the email server itself or merely be a way for emails to authorize access to web pages/documents like Google Cloud. Because what little I do remember of OAUTH required user interaction. One of the things I hated about it, when trying to get data imported into a webapp from a third-party server.
 
Bob Winter
Ranch Foreman
Posts: 103
7
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Well, at least when one looks at Thunderbird it seem to work like this: When you set a new gmail account a browser window opens that request access to some specific scopes. If one grants access Thunderbird ends up having a refresh token which somehow is used to generate access tokens so Thunderbird can access the users mailbox.
From what I know is that for get a new fresh access token from a refresh token you also need the google credential client secret - which most likely is only stored on Mozillas servers (or whoever is now in charge of Thunderbird).
So when the short time access token times out there has to be some communication between Thunderbird and the Mozilla servers to get a new access token. As Mozilla hopefully doesn't store users tokens to make this work Thunderbird has to send the refresh token. A Thunderbird client cannot do a refresh on its own as it would need a legit client secret - which would anyone who finds it enable to abuse it with full extend in the limits of accessible scopes.
 
Tim Holloway
Saloon Keeper
Posts: 22011
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Oooh. Now you're beginning to awaken old nightmares. JavaMail since version 1.5 has built-in OAUTH authentication. Just pass the token in as one of the mailer properties.

The ugly part as I remember it is that it required one to obtain the token via a manual web request - there was no way for automated code to get a token that I discovered - at least back then. And the token eventually expires, but that was the least of my worries.
 
Stephan van Hulst
Saloon Keeper
Posts: 11923
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

As I said that I'm new to building back-end web apps, can you maybe recommend some resource (tutorial, blog etc) on how to actually implement OAuth? By your comments, you seem to have an in-depth knowledge


Normally the idea is that you use the service's client API to access the service and let it handle OAuth for you. All you need to do is configure a client ID (and in the case of a web application, a client secret). For instance, to use Gmail you create an instance of the Gmail class that is based on a GoogleAuthorizationCodeFlow. If you were writing your own servlets instead of using a framework like Spring, you could make it even easier by just implementing AbstractAuthorizationCodeServlet and AbstractAuthorizationCodeCallbackServlet.

However, seeing as you're using Spring Boot, you can probably let the Spring Security module deal with all the OAuth stuff. Spring Security contains an OAuth2 provider for Google APIs. Check out the documentation here and the tutorial here.

You can probably get the access token from Spring Security some way after the user has logged in. Then you create an instance of the Gmail class like this:

This should work just fine, because Spring should be handling the refreshing of the token for you.

How to set (programmatically) mail server configuration parameters for each of this mail service, how do they differ?


After the user has logged in using OAuth through one of the Spring OAuth2 providers, you can probably get the provider type from Spring Security (see the two-providers example at the bottom of the tutorial). Based on the provider type, you can provide an implementation of an interface you designed that allows you to send an e-mail. For instance, the interface could look like this:

The Gmail implementation could look like this:
 
Tim Holloway
Saloon Keeper
Posts: 22011
151
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:You can probably get the access token from Spring Security some way after the user has logged in.



And here's the rub. OAUTH is designed to provide "Single Sign-On" across sites. But you have to prime the pump by signing on to an OAUTH participant. Because it's "Single" signon, not "No" signon.

But a larger question needs addressing. This is supposed to be Spring Boot and Android??? Android's Dalvik is not the same thing as a JVM. Spring Boot doesn't run on Android.
 
Stephan van Hulst
Saloon Keeper
Posts: 11923
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:OAUTH is designed to provide "Single Sign-On" across sites. But you have to prime the pump by signing on to an OAUTH participant.


I'm not sure what you mean by this. Yes it's single sign-on. So? User signs in to Google and if Google has allowed our mail service to access Gmail for that user, we can access Gmail.

But a larger question needs addressing. This is supposed to be Spring Boot and Android??? Android's Dalvik is not the same thing as a JVM. Spring Boot doesn't run on Android.


Mike is supposed to use Retrofit2 to send REST requests from Android to the Spring service, and the Spring service uses OAuth to access Gmail.
 
Mike Gosling
Ranch Hand
Posts: 101
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
First of all, thank you everyone for finding spare time to post your answers And thank you Stephan for detailed answer/tutorial (finally to see what Spring Security offers out of the box)!

Tim Holloway wrote:You basically just have to set all of the properties with values appropriate to the mailserver that you will be sending through. Same as with gmail, but each of those services has their own host address, port, security prototocols and connection credentials.


Yes, this sounds reasonable. So for each User, which has different mail service provider, different credentials. For now at least, I would like for Gmail users only to enable sending/receiving email without them having to to check "Enable less secure apps". For other ones I'll see later what to do.

Stephan van Hulst wrote:Mike is supposed to use Retrofit2 to send REST requests from Android to the Spring service, and the Spring service uses OAuth to access Gmail.


Correct!

However I'm kind confused about direction I should go. For now (before I move all the UI to Android), my goal is to send/receive emails using JavaMail API, by using OAuth (in order to avoid to check "Enable less secure apps"), which JavaMail API already supports, (as per Tim Holloway's previous post). Please correct me if I'm wrong.



By following JavaMail's OAuth support instructions, there are few steps I took, to obtain access token:

1. Went to Google Developers Console
-Made some app (marked as "Other") and got Client_ID and Client_Secrete

2. Run python script. oauth2.py program to generate a Refresh Token, from which you I generated an Access Token. Essentially, result I got looked something like this:



Now, since I need that JavaMail library to work somehow, I followed this tutorial. I adjusted the code so in my app it looks like this:




Of course, JavaMailAuthenticationException hit me like a rock. What I'm I doing wrong here? Do I have to use Spring Boot's OAuth classes and methods in order to make this work or I only need access_token?
And, that token is valid for one hour, how to do use refresh token to get new access_token during runtime?

Once again thank you all for your patience and time to read and hopefully answer on these posts
 
Stephan van Hulst
Saloon Keeper
Posts: 11923
253
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry, I missed your reply. Did you manage to make progress in the meantime?
 
Mike Gosling
Ranch Hand
Posts: 101
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Sorry, I missed your reply. Did you manage to make progress in the meantime?



No problem Stephan, thank you very much for keeping an interest. Actually I did make progress, I'll post thorough answer (more like a tutorial of steps) here, as soon as I find some time. They are killing us at university
 
Whatever. Here's a tiny ad:
ScroogeXHTML 8.7 - RTF to HTML5 and XHTML converter
https://coderanch.com/t/730700/ScroogeXHTML-RTF-HTML-XHTML-converter
    Bookmark Topic Watch Topic
  • New Topic