GCC-High Endpoints do not work #284
Comments
|
What is the |
|
After clearing everything out and trying a vanilla install and config in O365Interactive (with .us EWS), here is the log. Thunderbird which I am using for testing just sits there on "Sending login information" while DavMail status is < ******** then it times out eventually. 2023-04-22 06:34:59,620 DEBUG [JavaFX Application Thread] davmail.exchange.auth.O365InteractiveAuthenticatorFrame - Location starts with redirectUri, check code Then when trying in modern mode - in just fails as it immediately refers to login.microsoftonline.com instead of .us. When trying in manual mode - I can get the auth code and submit it back to DavMail but then when it actually goes to login again the logs reflect that it is attempt to pass the token to login.microsoftonline.com again which fails 2023-04-22 07:17:36,970 DEBUG [ImapConnection-59996] org.apache.http.impl.execchain.MainClientExec - Opening connection {s}->https://login.microsoftonline.com:443 |
|
@dsavagevaaxa, do you have a setting for I recommend that you try changing that setting, if you're not already doing so. I also recommend that you keep using O365Manual for now and only look at trying other modes once you get O365Manual working. That said, I do see places in the code where Do you feel up to compiling your own version of DavMail? |
|
I wasn"t aware of the https://outlook.office365.us endpoint so the default .com suffix is hardcoded for O365 in some places. => need to review code to make it compatible with non standard tenants |
|
At least two urls are hardcoded:
Can you confirm those would be the same but with .us suffix ? |
|
it seems that Office 365 GCC-High endpoints use outlook.office365.us instead of outlook.office365.com. You can find a list of Office 365 U.S. It is possible that the program is using a default endpoint configuration. You may need to modify the configuration file to include the correct endpoint. I suggest checking the documentation for DavMail. |
|
@mguessan recent commits, like 99a4944, look relevant to this issue. I tried searching the web for “davmail gcc high”, as well as the SourceForge discussions and mailing list, but I didn’t find anything. Is someone paying you to implement it now? Is the “audit” mentioned in other recent commits related to the GCC High support work? Just curious. Thanks for your work on this project in any case! |
|
@lhindir indeed this comment is related to GCC High (non .com), I refactored the code to regroup all reference to actual O365 urls in a single place (getO365LoginUrl) Of course I don't have an actual US gov O365 mailbox so I can't confirm changes are working as expected. The - yet to be documented - way to try this is to set a new property in davmail.properties:
|
This comment was marked as resolved.
This comment was marked as resolved.
|
@RickyDoug looks good, just please repost without the code= value, this is a sensitive token. |
|
Thanks. Hard to tell what does and doesn't need to be sanitized. It's been cleaned up. |
|
So do you have anything after check code ? |
|
A user’s organization needs to allow Davmail’s client ID via portal.azure.us for OAuth2 to work before this can be tested, right? |
|
With default DavMail clientid and redirecturi yes, with Microsoft native application clientId no, see #273 |
|
Sorry I haven't gotten to test this yet, but i managed to hurt my back and haven't been able to get back to the machine...but nothing after the check code. Also, I managed to get Outlook's Client ID and that used to work fine with the normal .com... no admin changes needed. Hopefully next week I'll be able to get back to this. And should this still be hitting a .com URL? |
|
I believe I am running into the same issue; I tried a trunk release and set |
|
@kenh wrote:
Well, it wouldn't hurt. 🙂 I recommend creating a wire debug log using a build of the latest GitHub master branch, as described here: https://davmail.sourceforge.net/faq.html#Feedback If you prefer to edit your properties file yourself, put the following line in your properties file: Quit and restart the DavMail application. Then have your mail program connect to DavMail and try to authenticate again. DavMail will then create a detailed log of what it is doing. This log will be stored in a |
|
In addition to the above I would suggest you try with Outlook clientId: The default DavMail clientId is registered on main O365/EntraID infrastructure, and of cours I can't create one on .us tld |
|
I am able to get this to work (version 6.0.1-3390) with my organization which is using GCC-High. I have to edit the domains to .us in these files before compiling: In my config file, I have to specify the tenantId and use this pair (for Gnome Evolution): If I try these: the link I get to obtain the token url redirects me to a page with this error: "AADSTS165000: Invalid Request: The request tokens do not match the user context." The 6.2.2 versions work, but I keep getting asked to open a browser to get a new token every day and once for each service (except imap & smtp-- those only need the token updated once). |
|
Still does not work in 6.2.2 -- though possibly only for clients that use third party authentication sites. When I observe the wire traffic, I still see an office365.com URL bumming around. I blame: I edited source to fix all .com to .us and did not try to confirm my assumption above. This also included changing graph.microsoft.com to graph.microsoft.us since I don't know what it is used for and you cannot change it via property...I was incredibly frustrated by this time (mostly due to java stupidities). There is a graph.microsoft.us variant hostname so I assume it should be usable if it is used in my/someone's configuration. I set all of the following properties before resorting to changing the source. I have not tried to delete the urls to see if the tls would obviate the need to change some of them. |
Is that exactly what you had set, @SethRobertson ? Because that should have been I think you may also need to set |
|
I have a few commits post release on this, please make sure you get the latest trunk build from github home page. In addition this is still considered experimental, I have no way to actually test GCC high endpoints. |
|
I am essentially up to date on trunk/master (I don't have the CRLF=>LF change). I did mess up tls instead of tld, but AFAIK the Settings.OUTLOOK_URL does not reflect those changes. So the lines identified by this grep will NOT change: rg '^[^/]*(OUTLOOK|O365)_URL' src/java I am 99% sure that the "resource" lines are absolutely wrong--when I snooped the traffic in the logs I saw resource in a web request with a .com address. The O365_URL and other OUTLOOK_URL ones I am only extremely suspicious of. |
|
Ah, yes, I see what you mean, @SethRobertson. I have opened PR #380 to change the instances of Would you be willing to test the changes in 919ec03 ? As @mguessan has said, he has no way of testing GCC high endpoints (and neither do I!), so you are our best hope to finally get this issue resolved. Would you be willing to compile davmail from source? There are two cases to test. One case is only setting |
|
@mguessan : I see |
|
I tested commit e6443a4. In all cases I had the davmail.oath.redirectUri set to "urn:ietf:wg:oauth:2.0:oob" With the following properties set, it WORKED. davmail.loginUrl=https://login.microsoftonline.us/ Without davmail.url set, it did NOT work. 2025-01-07 04:14:47,283 ERROR [ImapConnection-38090] davmail - Invalid URL: null With only davmail.tld set, it did NOT work. davmail.exception.DavMailException: Invalid URL: null With davmail.tld and davmail.url set, it DID work: davmail.tld=us I went back to the .tld-only configuration and entered the settings page. The URL was grayed out and I could only see the end of the Exchange URL. However the end looked good: ok.office365.us/EWS/Exchange.asmx |
|
Thanks, @SethRobertson ! I think we're getting pretty close here. Would you be willing to try one more change to That should change the default value of the |
|
Hi @esabol, thanks for taking a look at this. There seem to be numerous additional direct references to the properties for which your PR (#380) introduces high-level accessors. I think I got most of the critical ones (ignoring tests, as well as Expand diff---
src/java/davmail/Settings.java | 1 +
src/java/davmail/exchange/ExchangeSessionFactory.java | 6 +++---
src/java/davmail/exchange/auth/O365Authenticator.java | 2 +-
.../davmail/exchange/auth/O365InteractiveAuthenticator.java | 2 +-
src/java/davmail/exchange/auth/O365ManualAuthenticator.java | 2 +-
.../davmail/exchange/auth/O365StoredTokenAuthenticator.java | 2 +-
src/java/davmail/ldap/LdapConnection.java | 2 +-
src/java/davmail/ui/SettingsFrame.java | 6 +++---
8 files changed, 12 insertions(+), 11 deletions(-)
diff --git a/src/java/davmail/Settings.java b/src/java/davmail/Settings.java
index dccce122..11a59a10 100644
--- a/src/java/davmail/Settings.java
+++ b/src/java/davmail/Settings.java
@@ -44,6 +44,7 @@ public final class Settings {
public static final String OUTLOOK_URL = "https://outlook.office365.com";
public static final String O365_URL = OUTLOOK_URL+"/EWS/Exchange.asmx";
+ // FIXME
public static final String GRAPH_URL = "https://graph.microsoft.com";
public static final String O365_LOGIN_URL = "https://login.microsoftonline.com/";
diff --git a/src/java/davmail/exchange/ExchangeSessionFactory.java b/src/java/davmail/exchange/ExchangeSessionFactory.java
index de3e45cf..fc2d3958 100644
--- a/src/java/davmail/exchange/ExchangeSessionFactory.java
+++ b/src/java/davmail/exchange/ExchangeSessionFactory.java
@@ -88,7 +88,7 @@ public final class ExchangeSessionFactory {
* @throws IOException on error
*/
public static ExchangeSession getInstance(String userName, String password) throws IOException {
- String baseUrl = Settings.getProperty("davmail.url");
+ String baseUrl = Settings.getO365Url();
if (Settings.getBooleanProperty("davmail.server")) {
return getInstance(baseUrl, userName, password);
} else {
@@ -277,7 +277,7 @@ public final class ExchangeSessionFactory {
if (session.isExpired()) {
ExchangeSession.LOGGER.debug("Session " + session + " expired, trying to open a new one");
session = null;
- String baseUrl = Settings.getProperty("davmail.url");
+ String baseUrl = Settings.getO365Url();
PoolKey poolKey = new PoolKey(baseUrl, userName, password);
// expired session, remove from cache
synchronized (LOCK) {
@@ -301,7 +301,7 @@ public final class ExchangeSessionFactory {
* @throws IOException if unable to access Exchange server
*/
public static void checkConfig() throws IOException {
- String url = Settings.getProperty("davmail.url");
+ String url = Settings.getO365Url();
if (url == null || (!url.startsWith("http://") && !url.startsWith("https://"))) {
throw new DavMailException("LOG_INVALID_URL", url);
}
diff --git a/src/java/davmail/exchange/auth/O365Authenticator.java b/src/java/davmail/exchange/auth/O365Authenticator.java
index b7348c64..e738b037 100644
--- a/src/java/davmail/exchange/auth/O365Authenticator.java
+++ b/src/java/davmail/exchange/auth/O365Authenticator.java
@@ -123,7 +123,7 @@ public class O365Authenticator implements ExchangeAuthenticator {
// common DavMail client id
String clientId = Settings.getProperty("davmail.oauth.clientId", "facd6cff-a294-4415-b59f-c5b01937d7bd");
// standard native app redirectUri
- String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.O365_LOGIN_URL+"common/oauth2/nativeclient");
+ String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.getO365LoginUrl()+"common/oauth2/nativeclient");
// company tenantId or common
tenantId = Settings.getProperty("davmail.oauth.tenantId", "common");
diff --git a/src/java/davmail/exchange/auth/O365InteractiveAuthenticator.java b/src/java/davmail/exchange/auth/O365InteractiveAuthenticator.java
index 2c729e16..82980016 100644
--- a/src/java/davmail/exchange/auth/O365InteractiveAuthenticator.java
+++ b/src/java/davmail/exchange/auth/O365InteractiveAuthenticator.java
@@ -101,7 +101,7 @@ public class O365InteractiveAuthenticator implements ExchangeAuthenticator {
// common DavMail client id
final String clientId = Settings.getProperty("davmail.oauth.clientId", "facd6cff-a294-4415-b59f-c5b01937d7bd");
// standard native app redirectUri
- final String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.O365_LOGIN_URL+"common/oauth2/nativeclient");
+ final String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.getO365LoginUrl()+"common/oauth2/nativeclient");
// company tenantId or common
String tenantId = Settings.getProperty("davmail.oauth.tenantId", "common");
diff --git a/src/java/davmail/exchange/auth/O365ManualAuthenticator.java b/src/java/davmail/exchange/auth/O365ManualAuthenticator.java
index 1fcdef9c..1bbe2ce9 100644
--- a/src/java/davmail/exchange/auth/O365ManualAuthenticator.java
+++ b/src/java/davmail/exchange/auth/O365ManualAuthenticator.java
@@ -88,7 +88,7 @@ public class O365ManualAuthenticator implements ExchangeAuthenticator {
// common DavMail client id
final String clientId = Settings.getProperty("davmail.oauth.clientId", "facd6cff-a294-4415-b59f-c5b01937d7bd");
// standard native app redirectUri
- final String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.O365_LOGIN_URL+"common/oauth2/nativeclient");
+ final String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.getO365LoginUrl()+"common/oauth2/nativeclient");
// company tenantId or common
String tenantId = Settings.getProperty("davmail.oauth.tenantId", "common");
diff --git a/src/java/davmail/exchange/auth/O365StoredTokenAuthenticator.java b/src/java/davmail/exchange/auth/O365StoredTokenAuthenticator.java
index 05b2e745..bf08f838 100644
--- a/src/java/davmail/exchange/auth/O365StoredTokenAuthenticator.java
+++ b/src/java/davmail/exchange/auth/O365StoredTokenAuthenticator.java
@@ -63,7 +63,7 @@ public class O365StoredTokenAuthenticator implements ExchangeAuthenticator {
// common DavMail client id
final String clientId = Settings.getProperty("davmail.oauth.clientId", "facd6cff-a294-4415-b59f-c5b01937d7bd");
// standard native app redirectUri
- final String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.O365_LOGIN_URL+"common/oauth2/nativeclient");
+ final String redirectUri = Settings.getProperty("davmail.oauth.redirectUri", Settings.getO365LoginUrl()+"common/oauth2/nativeclient");
// company tenantId or common
String tenantId = Settings.getProperty("davmail.oauth.tenantId", "common");
diff --git a/src/java/davmail/ldap/LdapConnection.java b/src/java/davmail/ldap/LdapConnection.java
index aab33e0b..6058d2e4 100644
--- a/src/java/davmail/ldap/LdapConnection.java
+++ b/src/java/davmail/ldap/LdapConnection.java
@@ -943,7 +943,7 @@ public class LdapConnection extends AbstractConnection {
objectClasses.add("organizationalUnit");
Map<String, Object> attributes = new HashMap<>();
attributes.put("objectClass", objectClasses);
- attributes.put("description", "DavMail Gateway LDAP for " + Settings.getProperty("davmail.url"));
+ attributes.put("description", "DavMail Gateway LDAP for " + Settings.getO365Url());
sendEntry(currentMessageId, BASE_CONTEXT, attributes);
}
diff --git a/src/java/davmail/ui/SettingsFrame.java b/src/java/davmail/ui/SettingsFrame.java
index 5a743764..e89f7ca1 100644
--- a/src/java/davmail/ui/SettingsFrame.java
+++ b/src/java/davmail/ui/SettingsFrame.java
@@ -173,12 +173,12 @@ public class SettingsFrame extends JFrame {
modeComboBox.setToolTipText(BundleMessage.format("UI_"+selectedItem+"_HELP"));
if (selectedItem != null && selectedItem.startsWith("O365")) {
urlField.setEnabled(false);
- urlField.setText(Settings.O365_URL);
+ urlField.setText(Settings.getO365Url());
} else {
urlField.setEnabled(true);
}
});
- urlField = new JTextField(Settings.getProperty("davmail.url"), 20);
+ urlField = new JTextField(Settings.getO365Url(), 20);
popPortField = new JTextField(Settings.getProperty("davmail.popPort"), 4);
popPortCheckBox = new JCheckBox();
popNoSSLCheckBox = new JCheckBox(BundleMessage.format("UI_NO_SSL"), Settings.getBooleanProperty("davmail.ssl.nosecurepop"));
@@ -596,7 +596,7 @@ public class SettingsFrame extends JFrame {
*/
public void reload() {
// reload settings in form
- urlField.setText(Settings.getProperty("davmail.url"));
+ urlField.setText(Settings.getO365Url());
popPortField.setText(Settings.getProperty("davmail.popPort"));
popPortCheckBox.setSelected(Settings.getProperty("davmail.popPort") != null && !Settings.getProperty("davmail.popPort").isEmpty());
popNoSSLCheckBox.setSelected(Settings.getBooleanProperty("davmail.ssl.nosecurepop"));
--
2.47.1
With that, But it would be better to modify the internal Settings API so that consumers cannot access |
|
I think I got most of those in my PR, @lhindir, but I'll double-check. I'm not sure the instances in SettingsFrame.java need to be changed? I think they're just providing the default values for the GUI? Do you think a |
I generated that diff after checking out esabol@6e9f4fd and making changes, so there shouldn’t be overlap.
There’s no reason not to change them and leave the door open for the UI to misrepresent the application state, I think. Like I said, we shouldn’t be mixing raw references to the properties with the new accessors, and maybe there is a way to prevent this at compile time. Ideally @mguessan can chime in with his thoughts so you don’t waste too much effort on an implementation that won’t be merged.
No idea honestly. I’m not sure what the graph server is used for. But I would bet that whatever is using it is broken for GCC High if we don’t handle it like we do the other URLs. Thanks! |
|
@lhindir wrote:
Oh, OK. Thanks!
It's some new Microsoft-proprietary API for interacting with Microsoft Cloud services, basically retrieving mail and stuff, I think. I don't think DavMail currently fully supports it (because the API may not support everything DavMail needs?), but it's in the works.... What I'm not clear on is if GCC High supports this API or not, but it probably does, if I had to guess. Fwiw, graph.microsoft.us exists, but graph.microsoft.cn doesn't. |
|
Ah, thanks. I would guess GCC High supports it and if it doesn’t now it probably will in future. I just threw it in there since I figured as long as we’re getting away from hardcoding URLs, we might as well fix them all. |
Trying to get DavMail to work with Office 365 GCC-High endpoints. GCC-High uses outlook.office365.us rather than outlook.office365.com. I cannot seem to get this to function even after passing a valid token. The logs indicate that the program continues to target outlook.office365.com rather than the .us endpoint.
The text was updated successfully, but these errors were encountered: