steve labar

Ranch Hand
+ Follow
since Sep 10, 2008
Merit badge: grant badges
For More
Cows and Likes
Cows
Total received
0
In last 30 days
0
Total given
0
Likes
Total received
1
Received in last 30 days
0
Total given
0
Given in last 30 days
0
Forums and Threads
Scavenger Hunt
expand Ranch Hand Scavenger Hunt
expand Greenhorn Scavenger Hunt

Recent posts by steve labar

I have been trying for a while now to get a functional proxy going for http requests with no luck. I am currently using the http core Apache library as well as the http client to make it all work. Setting up the proxy to handle http was quite easy. However, as soon as I try to add anything that is coming from https the application fails. I think that part of my problem is that I'm still not entirely sure how the process is supposed to be handled. Initially, I was thinking that you set up a socket connection listen on port 8080 (or whatever port of your choosing) and as the browser sends the http requests the socket recognizes those requests are you as the programmer take those requests and making them and pass the response back to the browser. Problem is if my socket is a standard socket connection listening on port 8080 is handled fine. As soon as an https request comes in I received CONNECT request.once I see that connect request I change my clients on my proxy to send to the server via SSL and send back a response to the browser saying the handshake was complete. The problem is the next request that comes from the browser which I'm assuming is encrypted the socket closes. initially I thought that when I see the connect request I would change the socket to an SSL socket however what was happening then was if anything else from the browser came over the 8080 sslsocket was in clear text the SSL socket would throw a plaintext exception. So I'm thinking that the proxy server does not have to change the connection to SSL and simply just has to pass along the encrypted data to the server. Does this sound correct? Should I be able to simply create a standard connection on port 8080 and be able to handle both http and https?
If that is the case than though no cert will be needed between the browser and my proxy. However i think if the browser is sending https encrypted data its going to want to do a handshake and that cannot be done with standard socket. So needless to say im very confused!
I am trying to create a HttpsServer/Client so that I can create a proxy to examine traffic coming from the browser to the server. These types of tool are invaluable to people who test web application security. I have decided to use httpclient to send the requests and httpcore components for my server. At the moment I'm simply trying to establish the ssl socket connection between the browser and the server on port 8080. I have read all over and still cannot seem to get this to work. Here are the steps I did up to this point:

1. Created a CA cert with keytool and added it to file called cacerts

2. I added this cert to the firefox browser instance listening on port 8080

3v . In my code i do the following to call that cert in the server code



Then when I call the accept on the socket as seen below i get the following exception:

I/O error initialising connection thread: No available certificate or key corresponds to the SSL cipher suites which are enabled.
javax.net.ssl.SSLException: No available certificate or key corresponds to the SSL cipher suites which are enabled.
at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.checkEnabledSuites(Unknown Source)
at com.sun.net.ssl.internal.ssl.SSLServerSocketImpl.accept(Unknown Source)
at DefaultHttpServer$RequestListenerThread.run(DefaultHttpServer.java:151)





I think it is saying that the ciphers from the browser (client) does not have same ciphers? I have that cert in its configuration settings though. I honestly don't understand what I should be doing. Why the heck does java make SSL such a pain in the ass!


I read on a site the following explanation on how the proxy should handle certs

what I did for the proxy was got the client to trust a CA cert of my own.The mitm-proxy would then use that cert to generate whatever server certs are needed (on demand). the advantage of using a CA cert is that you will be able to mitm connections to new servers without having to get it to trust new certs you made up just now.



My questions are:

1) any idea where i'm going wrong on trying to establish the ssl socket?

2) that explanation of how the proxy"server" should handle certs is this how i'm approaching it? i have a self-signed cert in my trustore and in browser


Update 11/13
------------------------

I tried a few things and still no luck!. This is what I did

C:\Users\Steve>keytool -genkey -alias serverprivkey -keystore privateKey.store

Then I copied this file privateKey.store from my user directory over to my project folder and did the following changes in my code:

KeyStore ks = KeyStore.getInstance("JKS");
ks.load(new FileInputStream("privateKey.store"), "pass123".toCharArray());

KeyManagerFactory kmf = KeyManagerFactory.getInstance("SunX509");
kmf.init(ks, "pass123".toCharArray());

I know it is correctly grabbing that file because if passwords are wrong i get exceptions. However, i'm still getting that same exception. Any ideas what to try next?
I have been haunted for some time now trying to get my custom proxy to properly handle when the browser sends a CONNECT request. In order to keep it simple let me explain how I handle the process. Maybe at that point someone can help clarify what I'm doing wrong.


1. Create server with ServerSocketChannel on port 8080.
2. Bind that ServerSocketChannel to a Selector which essentially allows for non-blocking while the server waits for a request from port 8080.
3. As soon as I set my browser to port 8080 and send the request https://google.com it notifies the selector something sent to port 8080.
4. I get that request and see its a CONNECT so i immediately create a response "Connection Established" (request and response i send and receive are below)

Request from browser:
CONNECT google.com:443 HTTP/1.1
User-Agent: Mozilla/5.0 (Windows; U; Windows NT 6.1; en-US; rv:1.9.2.18) Gecko/20110614 Firefox/3.6.18 GTB7.1
Proxy-Connection: keep-alive
Host: google.com


Response I send back to browser over my normal socket:
HTTP/1.1 200 Connection established\r\nProxy-connection: Keep-alive\r\n\r\n

5. Now I wait for something to be sent from browser I was assuming at this point a SSL request would be sent but nothing ever gets sent. I'm starting to think its because I have not established a SSL handshake with the browser so its not going to send a SSL message over that default created socket. Do you think I need to close that socket over port 8080 and establish a new SecureSocket on port 8080 right before I send the connection established response back to browser? This is my next step.I know that the browser needs to send me more data after the initial CONNECT. I don't have enough data with just the CONNECT to go to server yet. I'm thinking it than needs to send me another request something like the following in SSL:

GET /
Host: google.com

Once I get something like that then I can go establish my secure socket connection with the server and get back the response to send back to browser.

What you think on the right track? Its just that I get no additional message after i send connection established.
I'm having a difficult time trying to optimize the adding of a large file to a JTextPane. I have searched everywhere on the internet and cannot seem to find anything that looks feasible. One site mentions removing the JTextPane's document while insertions are being done to avoid any unnecessary updates. I tried this but it did not seem to help much.

What I'm thinking of doing is creating the zone model where i load a chunk of the data and listen for the user to reach the end of the scrollbar. At that point I add another chunk to the pane.

I have that functionality working fine. However, I would like to make the insertion of text much more efficient. Anyone have any tips how to speed things up?

- I grab text from file and put into a string > mb - this is fine

- I then call pane.settext(string) -> this is where things really hang.
12 years ago
I added the adding of the data to a Runnable class just to make sure it was not a thread issue. Still get same issue so I know its not that



Then inside the update method that is threaded because its spawned by the Observer I added the following line right before adding data to model. Noty sure if this is necessary but I have heard anything on the SWIng should be done on EDT thread.

12 years ago
After adding a TableRowSorter to a table and its corresponding model any corresponding adds specifically at firetabletablerowsinserted cause exceptions. It is clear from testing that the GetRowCount() is returning a value past the models range. However it does not make sense to me how to continue to add values to the table after a sorter or filter has been added?

As an example, I set the row filter before adding anything to the table then add a value to the table with the following calls in my model:



The rowcount is of size 1 and the exception is thrown:

java.lang.IndexOutOfBoundsException: Invalid range at javax.swing.DefaultRowSorter.checkAgainstModel(Unknown Source) at javax.swing.DefaultRowSorter.rowsInserted(Unknown Source) at com.gui.model

If I do the same steps without first adding the sorter everything is fine. I assumed that possibly I needed to notify the model that the sorter may have made changes and tried the following but still returns an exception:




So, to summarize I have tried to call the this.fireTableRowsInserted(this.getRowCount(), this.getRowCount()); with both the sorter rowcount and that fails. So, I also tried to check in the update when adding to the table to see if a sorter exists. If the sorter exists I try to update the sorter itself. This does work like the following example. Problem I have with this is two fold.

1. Its calling update on the entire table. Once the table gets large its updating sort for the entire table. When only one row has been added. It really should only need to call sort on the new row. But any time I get specific like sorter.rowsInserted(firstRow, endRow) I get invalid range exceptions.

2. You would think that by updating the model you would niot need to worry about handling another model. I was hoping it would delegate sorting once the model its sorting is updated?




If anyone could try to clarify how best to handle this I would be very grateful. I will post the entire Update method below this method essentialy is the update inside my tablemodel class. All observers of the model add data to the table from this method by using Observer pattern.

12 years ago
ended up being that SSLResult when returned needs to call unwrap multiple times in a loop to get the data. One call to unwrap does not give you the full contents of the response!
I'm trying to establish a SSL connection to a site and return its full response back. For some reason no matter what SSL site I go to when I step through the code it seems like the first read grabs a chunk of data and all subsequent calls return 0. I did some research and it appears there are two cases you would get back an int value of 0 from a read:

1. The server is busy ,lag with internet,etc - I do not think this is the issue because the loop calls read for a good 60 seconds during that time always returns 0.


2. The ByteBuffers passed into read are full and therefore cannot be written to - When looking at buffer I added a capacity of 100000. This is well below the possible response being added. I can tell by the position that the buffer has plenty of room in it? This appears to be a problem with a number of sites I have tried so I know its not an isolated server issue. I was hoping someone could glance at my securesocket class see if you can see any blatant issue that I'm overlooking that could be causing the problem:

Most importantly the read command. I believe the establishing of the connection is working because i'm getting data back from server. It's just never the full response.


A few things I'm seeing:

my request to https://encrypted.google.com/ on first read returns the headers as seen below. Then I pass that data to the response handler that checks the response notices its not complete goes back and calls SecureSocket.read() again. All subsequent calls get a result of 0 back but I know there is more content maybe the content of the response? I'm just perplexed as to what I'm doing wrong.

In other sites I get even more of the response but never seems to get everything. In most cases after initial read everything is 0.

HTTP/1.1 200 OK
Date: Thu, 12 May 2011 04:17:50 GMT
Expires: -1
Cache-Control: private, max-age=0
Content-Type: text/html; charset=UTF-8
Content-Encoding: gzip
Server: gws
Content-Length: 9008
X-XSS-Protection: 1; mode=block

Thank you very much for taking the time to look at the code. I will redo how I parse the chunks and retry!
12 years ago
I have been trying for a week now to correctly use a socket to read the stream of a response from a server than simply parse out the GZIP response body portion and than place those bytes into the GZIPInputStream and get back my results as a string. I'm able to get it to unzip for certain responses however others fail? I think I'm incorrectly removing bytes from the stream maybe when I dechunk the response. The response is sent in a chunked format so I need to remove those bytes and put back together in order for the gzip file to be correct. So here is what I do first let me give you the gzipped test response I have created:

Gzip Unit Test

The contents of gzip.php which send that content gzipped up



Here is the response and the parsed content I get from the server,after parsing the content of response out and after cleaning up erroneous chunk data out:




I'm assuming I must be stripping out an important byte somewhere when I'm cleaning up the bytes here is how I get those bytes above:

To strip out the headers and body from response I do the following:



To me that process looks to be getting the correct content as shown by the bytes above chunked response body looks to be the content of response in bytes to me. Know because this particular response is chunked I need to remove the erronous chunked headers insinde that response body:



I have a sneaking suspicion this is where i'm stripping out maybe a byte or leaving a byte that should not be a part of the gzipped data. After this I send those bytes to the GZIP deflater below:



So whats happening is for a response like the one below after gzip deflates I get the following:



String after deflating and excetion thrown:



But a simple change of the s in Lights in response to Lighta like the following response:



I get back correctly unzipped with no exceptions? Can anyone give me any ideas about what may be wrong? I tried taking that same response content and place inside a file and gzip it and it unzipps just fine. I even tried to compare the bytes of the gzips from sever vs file and they are completely different for some reason. Even when the server one correctly unzips so I can't use that as a way to compare the bytes to see which byte from server may be missing?

This was what the response looks like in bytes from file:
From File which decompresses correctly:
[31, -117, 8, 0, 0, 0, 0, 0, 0, 0, -77, -55, 40, -55, -51, -79, -29, -27, -78, -55, 72, 77, 76, 1, -47, -71, -87, 37, -119, 10, 25, 37, 37, 5, -70, -87, -123, -91, -103, 101, -74, 74, -50, -7, 121, 37, -87, 121, 37, -70, 33, -107, 5, -87, 74, 10, -55, 16, -98, -83, 82, 73, 106, 69, -119, 62, 72, -77, -75, -126, -77, -121, 99, 80, -80, 107, -120, 109, 105, 73, -102, -82, -123, 18, -56, -112, -110, -52, -110, -100, 84, 59, -49, -68, -92, -4, 10, 5, 93, 5, -1, -46, -110, -100, -4, -4, 108, -123, -16, -44, 36, 5, -57, -28, -28, -44, -30, 98, 5, -97, -52, -12, -116, -110, 98, 0, -127, 62, 118, 36, 125, 0, 0, 0]

As you can see the one from server and file look very different so I can't use them to compare. I'm open for any and all ideas on what to try next.
12 years ago
So, here is the format of the bytes of the gzip data before attempting to pass it into the GZIPInputStream.

//-117, 8, 0, 0, 0, 0, 0, 0, 3, -78, 81, 116, -15, 119, 14, -119, 12, 112, 85, -56, 40, -55, -51, 81, 8, 8, 117, -14, -15, 116, 86, 80, -46, -43, -41, 15, 55, 118, -42, -41, 119, 9, 113, 81, -120, -16, 8, -15, -11, 81, 48, -44, 51, 80,
So, i'm hoping some of you guys can help me with my problem. Unfortunately I need to roll my own HTTPClient. I have spent quite sometime researching sockets and have a rough client working. It grabs text/html just fine. However, as soon as I try to get a response that is of Encoding-Type: gzip I cannot seem to get the ByteArray to unzip. I get the following error: java.io.IOException: Not in GZIP format

I know it has gotta be how i'm storing the zipped data that might be the issue. I have noticed that the header for content-type tells me the length should be 12342 however after getting all the data my ByteBuffer has a capacity of 13245. That seems a bit strange?

I was hoping you guys would not mind looking at my code see if it looks logically correct?



I have embedded comments in code to show what i'm doing any ideas about why my ByteBuffer.capacity would be larger than header states it should be? This is the same when I pull text/html responses the ByteBuffer.capacity is larger than content length? So, i'm hoping if I figure this out this will fix my GZIP issue!

So, I have found if I take the ByteBuffer and convert it into string and then get the length its correct? What is inside the ByteBuffer that's being stripped when its converted to as string?
I have two more strings overwritting eachother? what in the world is going on?

String three ="WEBRPTID";
String four = "USER";
HashMap<String,String[]> m = new HashMap<String,String[]>();
m.put(three, new String[]{"",""});
m.put(four, new String[]{"",""});
13 years ago
I had the following test collide and can't seem to explain it:

String one = "ctl00$ctl00$contentBody$cphMain$RadDockA$C$tbPassword";
System.out.println(one.hashCode());
String two = "ctl00$ctl00$contentBody$cphMain$RadDockA$C$tbLoginID";
System.out.println(two.hashCode() + "" +one.equals(two));
HashMap<String,String[]> m = new HashMap<String,String[]>();
m.put(one, new String[]{"",""});
m.put(two, new String[]{"",""});

The map overwrites the key? I just don't understand why? When I print out the hashcodes they are diferent and equals on these two should return false. Any ideas what is going on and how i can fix this in the future.
13 years ago