Logging HTTPS GET requests

[D Connor]

Hello

There is a java applet / jar file that opens a GUI where I can click buttons to download files.

I am looking to check the urls of these files and the HTTPS GET requests. There is probably an API so I would be looking for the URIs and also some other metadata like session IDs etc.

There seem to be some software like Fiddler, Wireshark but I need to use something more native to Windows or JAVA SDK/console.

Is there e.g. a way to check these HTTPS requests via a JAVA or Windows built-in diagnostic or debugging/logging tool?

There are also some reverse proxy tools but would prefer something more native.

Any idea?

Thanks!

[Tim Cooke]

Hello, Welcome to the Ranch!

Can you expand upon why WireShark and the like are unsuitable? You could also probably unpack the jar and run the class files through a decompiler to discover the URLs the app is using. Decompilers general result in Java files that are not easy to read and reason with but a String of the URL might be easy to find, or you might find it in a plain text config file.

Are you just trying to discover the backend services so you can bypass the little app you have?

[Tim Holloway]

I think there's a Windows version of Wireshark, actually.

If they're all going to a server that you control you can just look at the server's request log.

[Tim Cooke]

Sounded to me like the OP didn't own either the application or backend service. But I might be wrong, it's a good point.

[Matthew Bendford]

The term you're looking for is "tls terminator" - it's pretty much a proxy that "breaks" (or "terminates") the TLS by a simple MITM attack. The only requirement is that your client has a copy of the master root ca the tls terminator uses to create certs on the fly. That's pretty much how any big company networks are built to a) block employees from accessing the internet and b) to spy on them. If you work in such a company there's a specific clause about that in your contract.
On the other hand: Using BouncyCastle to do the certificate stuff it's easy to write a tls terminator proxy your own with not that much code required.

[D Connor]

Thanks, I had unzipped the jar and checked the files, I found some links and some documentation like API documentation but I would like to know the exact link of the file downloaded because I suspect it is something like domain.com/xxx-xxx-xxx-xxx (kind of the usual API links for specific data requests) hence I don't think it won't be documented like that in the jar files.

I will resolve to Wireshark and Fiddler if there is no other option but wondering if there is anything lighter in terms of messing with the system or having dubious EULA.

I came across to membrane-soa.org but it is not very plug and play.

I am actually surprised there is no built in logger within the Java SDK or Console etc. I think Apache has some tools too like Log4j but again was hoping for something native to Windows or Java (Windows has so many logging capabilities but I only managed to check pktmon which again is not plug and play at all).

[Paul Clapham]

D Connor wrote:I am actually surprised there is no built in logger within the Java SDK or Console etc.

The built-in logging for the Java JDK is found in the java.util.logging package. See for example a tutorial I found at https://docs.oracle.com/javase/10/core/java-overview-logging.htm or search for other tutorials.

I don't know why you favour native Java implementations, though. The native Java logging appeared in Java 4, I think, or at any rate a very long time ago, but it was universally ignored because superior implementations like log4j were already available.

[Stephan van Hulst]

Out of curiosity, what's dubious about the Fiddler/Wireshark EULAs? Fiddler is my go-to network debugger, and sometimes I will make a small side strio to Wireshark if I need a bit more detail/power. So far I've never worried about tge EULAs because they come pre-installed on my company system.

[Tim Holloway]

Logging has been around in Java for a long time in many flavors, including the built-in one (JULI). But logging isn't the same thing as snooping. The only time anything gets logged is if something explicitly requests to be logged.

If you want something under Linux that can see things like URL requests, you'd have to use a kernel-level tracing tool like DTrace.

[D Connor]

Stephan van Hulst wrote:Out of curiosity, what's dubious about the Fiddler/Wireshark EULAs? Fiddler is my go-to network debugger, and sometimes I will make a small side strio to Wireshark if I need a bit more detail/power. So far I've never worried about tge EULAs because they come pre-installed on my company system.

Yes, that's the thing, they don't come pre-installed on my system and I am not sure if they are free to use.
I will check the proposals, thanks!

[Stephan van Hulst]

You can evaluate Fiddler Classic for free for an unlimited trial period.

[Tim Holloway]

Wireshark is available for free on Linux, and in fact, is usually included in the standard distros. I can't vouch for it on Windows - some products are free on Linux but paid-only on Windows.

[Ron McLeod]

D Connor wrote:... I am not sure if they are free to use.

From Wireshark Frequently Asked Questions ...

Wireshark FAQ wrote:How much does Wireshark cost?
Wireshark is "free software"; you can download it without paying any license fee. The version of Wireshark you download isn’t a "demo" version, with limitations not present in a "full" version; it is the full version.

The license under which Wireshark is issued is the GNU General Public License version 2.  See the GNU GPL FAQ for some more information.

Wireshark FAQ wrote:Can I use Wireshark commercially?
Yes, if, for example, you mean "I work for a commercial organization; can I use Wireshark to capture and analyze network traffic in our company’s networks or in our customer’s networks?"

If you mean "Can I use Wireshark as part of my commercial product?", see the next entry in the FAQ.

[D Connor]

Mmm had a try with Wireshark, but based on the below article, Fiddler would be better as it supports:
1) https (thus needing decryption)
2) firewall/vpn present
3) hostname traffic (the local java applet would probably communicate via localhost address)
https://confluence.atlassian.com/kb/how-to-capture-http-traffic-using-wireshark-fiddler-or-tcpdump-779164332.html

Fiddler asks for email and usage which is a bit too invasive.
Wireshark warns that packet monitoring may not be permitted in some networks which makes it much more complicated to use these.

That's why I was initially looking for an as native as possible logger/debugger  
I am not a formal network technician to justify non native tools.

[Matthew Bendford]

In order to prevent actions for staff:

solution to your question: https://en.wikipedia.org/wiki/TLS_termination_proxy

answer to your question "why I don't find any logs?": mostly because what you searching for doesn'T get logged

// rest cut off for several reasons

[D Connor]

Thanks! Yes that is what seems the right tool. I tried Fiddler it is really too invasive, it installs certificates DO_NOT_TRUST_Fiddler in Windows. That's why I was hoping for a native solution. Can anyone recommend an easy to setup reverse proxy that supports HTTPS and localhost? Or any other JAVA monitoring method?

It would also help if I can force to open a SunAwtFrame via Chrome etc and then I can use Chrome web dev tools. Or any other workaround would be appreciated.

I considered IIS but I cannot get them in my Win copy.

[Stephan van Hulst]

Wait, you consider an application that asks you to install a certificate (with an obvious name to warn you about it) MORE invasive than an application that MITMs your HTTPS traffic using black magic?

[Matthew Bendford]

D Connor wrote:It would also help if I can force to open a SunAwtFrame via Chrome etc and then I can use Chrome web dev tools. Or any other workaround would be appreciated.

I have to hold back to not get mods angry to take actions against me - but I have to let lose this part:

Set up a VM with like 98 or ME or XP - something that supports something old enough like java5 or java6 which still supports applets - write some lines in which an applet access data on the server it got loaded from - and see for yourself: It won't show up in the browser dev tools.
TLS encryption is to prevent exactly THAT - that someone spies on your traffic - which the URL is itself a part of. If you want to break security you have to face what potential risk it comes with. If you don't want to accept this risk - then there's no way.

And again: THERE IS NO BUILT IN MAGIC BLACK BOX TOOL THAT LOGS WHAT'S HIDDEN IN ENCRYPTION! Get it ... it's as basic physics as everything falls down to center of gravity.

[D Connor]

Stephan van Hulst wrote:Wait, you consider an application that asks you to install a certificate (with an obvious name to warn you about it) MORE invasive than an application that MITMs your HTTPS traffic using black magic?

Well, that's why I was asking for something 'native' so that it can use the existing certificates as I believe Web Dev Tools do in Chrome. Any third party tool would be too invasive as I imaged.