Win a copy of Transfer Learning for Natural Language Processing (MEAP) this week in the Artificial Intelligence and Machine Learning forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Tim Cooke
  • Paul Clapham
  • Devaka Cooray
  • Bear Bibeault
Sheriffs:
  • Junilu Lacar
  • Knute Snortum
  • Liutauras Vilda
Saloon Keepers:
  • Ron McLeod
  • Stephan van Hulst
  • Tim Moores
  • Tim Holloway
  • Piet Souris
Bartenders:
  • salvin francis
  • Carey Brown
  • Frits Walraven

Proper Way to Use Data Connection in Filter

 
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

I am attempting to create a connection in my filter. I have set an attribute in the ServletContextListener for the DataConnection(Connection Pooling).
My question is do I have the Connection (setAttribute) set up correctly.



Also, when I am calling methods do I just pass the request object?
Or do it like so?



Also have a concern that the Connection is closed in the proper place.

if I use this in my method do I need to do a try-finally?

 
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
This doesn't look right at all. Why are you opening a connection in a filter? And why are you ignoring certain URLs? And why are you wrapping the request/response in a custom wrapper?

I don't think a filter is the right place to set up connections. And even if you really want to do this, you're doing too much in a single filter. Filters should have a clear, single responsibility.
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:This doesn't look right at all. Why are you opening a connection in a filter? And why are you ignoring certain URLs? And why are you wrapping the request/response in a custom wrapper?

I don't think a filter is the right place to set up connections. And even if you really want to do this, you're doing too much in a single filter. Filters should have a clear, single responsibility.



I found this in Stack Overflow:

In a Web app, each http request from a user runs in its own thread (or through a thread pool). Having a global connection object shared by all requests is not a good idea because only one thread at a time will be able to use it and it will become a bottleneck for your app.

The recommended setup is to use a connection pool (e.g. C3P0) that is initialized on server startup (either automatically through config or manually in your ServletContextListener). The connection pool will create and close the connections as needed. When you receive an http request (servlet's doPost or doGet), you then just have to get a connection from the pool and return it to the pool once you're done processing that request.

You can use a ServletFilter to automate that part. In your filter, before the call to chain.doFilter(), you get a connection from the pool and store it in a request attribute. After the call to doFilter(), you return it to the pool.
 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
It's good to use a connection pool. It's bad to manage the connection pool yourself.

You need to configure a DataSource in your web container or application server (e.g. Tomcat, WildFly) and you need to configure your persistence framework (e.g. Spring Data JPA, Hibernate), to use the connection pool configured in your application server. All of this can be done through config files.

What application server and persistence framework are you using? Note that instead of a high level persistence framework you can also use JDBC to access the connection pool directly.
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:It's good to use a connection pool. It's bad to manage the connection pool yourself.

You need to configure a DataSource in your web container or application server (e.g. Tomcat, WildFly) and you need to configure your persistence framework (e.g. Spring Data JPA, Hibernate), to use the connection pool configured in your application server. All of this can be done through config files.

What application server and persistence framework are you using? Note that instead of a high level persistence framework you can also use JDBC to access the connection pool directly.



This is what I use for connection pooling:

 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That looks like a bad idea to me. Rogue code can just change the mode from a completely unrelated part of the code and you'd be none the wiser. Why do you set the mode in the code? What is the difference between DB2WeberCustom and DB2WeberDEV?
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:That looks like a bad idea to me. Rogue code can just change the mode from a completely unrelated part of the code and you'd be none the wiser. Why do you set the mode in the code? What is the difference between DB2WeberCustom and DB2WeberDEV?



Two different servers. Production and Test.
 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yeah. What you're doing is wrong. It shouldn't be up to your application to decide what database connection to use.

The right way to do this is use one single logical name to get the data source from JNDI. Then in your web application container you can associate the logical name with a physical name that refers to the real data source that you configured in the web application container.

For instance, use "java:comp/env/jdbc/DB2Weber" to refer to the data source from your application. Then in the Tomcat instance (assuming you're using Tomcat) you're running on your local machine, you configure "java:comp/env/jdbc/DB2Weber" to refer to the DEV data source. When you deploy the application to the production server, it will use the production database if in that Tomcat instance you've configured "java:comp/env/jdbc/DB2Weber" to refer to the data source with the proper physical name.

What web application framework are you using, and what web application container will you be running the application in?
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Yeah. What you're doing is wrong. It shouldn't be up to your application to decide what database connection to use.



I still have the question. Do I need to do a try-finally to close the dc on the following code?

 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Yeah. What you're doing is wrong. It shouldn't be up to your application to decide what database connection to use.

What web application framework are you using, and what web application container will you be running the application in?



Rational Application Development(Web)
WebSphere Application Server running on an iSeries machine.
 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I'm going to assume you're using Java EE.

What Java EE version are you using? And are you using a JPA implementation (such as Hibernate) or are you using JDBC directly (writing SQL queries and sending them using the Connection class)?

Steve Dyke wrote:I still have the question. Do I need to do a try-finally to close the dc on the following code?


You shouldn't have that code at all. You should inject your data source/query builder into your DAO/repository classes using CDI and then close the connection when you're done handling the request. What to do exactly depends on what persistence framework you're using (see my questions above).
 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Also, how are you handling client requests? Are you writing custom servlets? Are you using some MVC framework? Are you using enterprise beans?
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Also, how are you handling client requests? Are you writing custom servlets? Are you using some MVC framework? Are you using enterprise beans?



My Java version is 1.7.
I am not using a formal framework but try to pattern the app using MVC.
I use JSP/jQuery on the client side. This makes requests to the servlets.
I use the sql Connection class(jt400.jar) to communicate with our AS400 data server.
I have connection pool set up on the WebSphere Application Server.
In my Deployment Descriptor there is a JNDI reference set up to the connection pool.
 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So you are implementing a custom HttpServlet to handle requests? What do the requests look like, are they like REST requests or SOAP requests or some custom format?
 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Steve Dyke wrote:I use JSP/jQuery on the client side. This makes requests to the servlets.


Wait... you're not accessing the database directly from a JSP, are you?
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:So you are implementing a custom HttpServlet to handle requests? What do the requests look like, are they like REST requests or SOAP requests or some custom format?



Servlet Context Listener:



Filter:



Client side:



Server side:



Data layer:

 
Stephan van Hulst
Saloon Keeper
Posts: 11881
253
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Your code would be a LOT simpler and robust if you used REST controllers and an IoC container. Then all you needed to do would be something like this:




When you make a request POST /QMSWebApp/api/foos/recalculateRandomConformityMatrix JAX-RS will automatically route your request to FooController.recalculateRandomConformityMatrix(), and CDI will inject the repository into your controller, and the data source into your repository.

If you need any data from the session, encapsulate it in a class and annotate the class with @SessionScoped. Your application container will keep track of the session and the related data for you. Then inject an instance of the class into a constructor or field using the @Inject annotation.
 
Saloon Keeper
Posts: 21960
149
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Whoah. Let's keep this simple and in keeping with accepted Best Practices.

Recapping what others have said:

1. Don't create a connection pool - or Connections! - yourself within the webapp. All JEE-compliant webapp servers have the ability to create and manage Connection Pools which can be attached to single webapps or shared between multiple webapps. That insulates the database particulars from the application code, leading to easier testing (just swap out databases in the server) and more portable coding.

2. The server will publish Connection Pools that it creates for a webapp as JNDI resources. It's a good idea to set up a ServletContextListener that locates that pool(s) via JNDI lookup and caches the pool interface as an Application-Scope object. Not that JNDI is that much overhead, but it's less trouble to pull the pool from cache. However, DO NOT CACHE CONNECTIONS! More on this later. The Pool will exist before the app starts and continue to exist until after all apps that use it shut down, so caching the pool interface is OK.

3. ABSOLUTELY NEVER store ANYTHING in a member of a Servlet or anything else that can get called by the servlet's thread. If you attach data to a servlet thread and the servlet returns from doGet/doPost or any other process function, the polluted thread will be returned to the thread pool and potentially randomly corrupt/crash the entire server. Remember - servlets are not programs and they do not run continuously. They're callable services.

4. NEVER save a Connection. Especially across servlet process calls. Connections are Interfaces, not Objects, and they are not guaranteed serializable. Obtain a Connection from the pool, process the database functions and return the Connection to the pool AS SOON AS POSSIBLE. This allows more efficient use of Connection resources, especially when there are many concurrent web requests.
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:Whoah. Let's keep this simple and in keeping with accepted Best Practices.

Recapping what others have said:

1. Don't create a connection pool - or Connections! - yourself within the webapp. All JEE-compliant webapp servers have the ability to create and manage Connection Pools which can be attached to single webapps or shared between multiple webapps. That insulates the database particulars from the application code, leading to easier testing (just swap out databases in the server) and more portable coding.



The connection pool is created on the WebSphere Application Server.

I have this class:



Tim Holloway wrote:2. The server will publish Connection Pools that it creates for a webapp as JNDI resources. It's a good idea to set up a ServletContextListener that locates that pool(s) via JNDI lookup and caches the pool interface as an Application-Scope object. Not that JNDI is that much overhead, but it's less trouble to pull the pool from cache. However, DO NOT CACHE CONNECTIONS! More on this later. The Pool will exist before the app starts and continue to exist until after all apps that use it shut down, so caching the pool interface is OK.



My Servlet Context Listener:



Is this correct so far?

If so in my classes that need to get remote data I would??

 
Tim Holloway
Saloon Keeper
Posts: 21960
149
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Allowing for my incredible ability to miss things, that looks OK to me. Main difference is that I probably would break down the connection fetch into multiple statements, since if any of the steps fails, it's hard to tell which one failed from a stack trace. But of course it's not supposed to fail.
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:Allowing for my incredible ability to miss things, that looks OK to me. Main difference is that I probably would break down the connection fetch into multiple statements, since if any of the steps fails, it's hard to tell which one failed from a stack trace. But of course it's not supposed to fail.



Are you referring to this?



And just to make sure I need to use the following any where I fetch the connection fetch:



One more question. Are the Connection dc instances scoped to the request?
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:Your code would be a LOT simpler and robust if you used REST controllers and an IoC container. Then all you needed to do would be something like this:




This certainly looks like cleaner code. However, I have not used annotation up to this point.
I tried to use this piece of code and even with reading several documents(maybe not the correct ones) I still do not understand the @Resource or @Produces. I get errors on these.
Do they need to be defined in my app? Are they a part of a library? Are they native?
Sorry to be unlearned in this respect. But I desire to learn and make my application better.
 
Tim Holloway
Saloon Keeper
Posts: 21960
149
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Yes, in answer to the first question. Second question, yes. Always explicitly close a datasource Connection. That's what returns it to the Connection Pool for re-use. Otherwise it can become an orphan and eventually the pool will run dry.

The Connection has no JEE scope. The reference(s) to the Connection are restricted to the scope of their execution paths as any normal Java variable would be. They are not controlled by the lifecycle of Request scope, and, as I said before, should never be kept in Session scope. Putting them in Application scope would simply be a waste of resources, and probably violate JEE standards, since I think that application scope objects, like session scope object are required to be serializable and Connections are not.
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:Yes, in answer to the first question.



I thank you for spending time to explain this to me and your sense that I need a more basic explanation than should be required.

This application is huge. I have had it deployed and work on it daily to improve it for the last 14 years. It has came a long way.
However, I have always felt the way I was connecting to the data was not working the best way. Now I know thanks to JavaRanch.

I have tried to break this line down into more statements but to no avail.

 
Tim Holloway
Saloon Keeper
Posts: 21960
149
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator


Assuming I havent screwed up too badly. The actual code generated should be no less efficient than the original statement, but if one of the operations fails (for example, returns null), the stack trace makes it more obvious where the fault was, since each sub-operation now has its own line number. Bonus, if you happen to need any other Application Scope objects, the context is already broken out.

And incidentally, the gratuitous use of "final" is just me being paranoid. Because, as they say, Constants - aren't and variables - won't.
 
Steve Dyke
Ranch Hand
Posts: 1985
1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:

4. NEVER save a Connection. Especially across servlet process calls. Connections are Interfaces, not Objects, and they are not guaranteed serializable. Obtain a Connection from the pool, process the database functions and return the Connection to the pool AS SOON AS POSSIBLE. This allows more efficient use of Connection resources, especially when there are many concurrent web requests.



Could I please get a little more clarification here?

In my Servlet Context Listener I use setAttribute to set up application level data.
Would this be acceptable in lieu of a connection for each constructor.

I also have instances where I use this approach if I have embedded SQL calls. I will wrap one of these Connections around all of the SQL code.

 
Would anybody like some fudge? I made it an hour ago. And it goes well with a tiny ad ...
Two software engineers solve most of the world's problems in one K&R sized book
https://coderanch.com/wiki/718759/books/Building-World-Backyard-Paul-Wheaton
    Bookmark Topic Watch Topic
  • New Topic