This week's book giveaway is in the Java in General forum.
We're giving away four copies of Event Streams in Action and have Alexander Dean & Valentin Crettaz on-line!
See this thread for details.
Win a copy of Event Streams in Action this week in the Java in General forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
programming forums Java Mobile Certification Databases Caching Books Engineering Micro Controllers OS Languages Paradigms IDEs Build Tools Frameworks Application Servers Open Source This Site Careers Other all forums
this forum made possible by our volunteer staff, including ...
Marshals:
  • Campbell Ritchie
  • Devaka Cooray
  • Liutauras Vilda
  • Jeanne Boyarsky
  • Bear Bibeault
Sheriffs:
  • Paul Clapham
  • Knute Snortum
  • Rob Spoor
Saloon Keepers:
  • Tim Moores
  • Ron McLeod
  • Piet Souris
  • Stephan van Hulst
  • Carey Brown
Bartenders:
  • Tim Holloway
  • Frits Walraven
  • Ganesh Patekar

How to modify and compile only few java files but not the many thousands and deploy?

 
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

This is basically a code development question that deals with frequent changes to one or more java files among many thousands.

A few years ago, as a software intern, I was given a large java / jsp code base and my mentor set me up using intellij IDE to build a WAR file for a web application (a student portal). I would then copy the War file to the tomcat container and launch it. I don't recall, but I recall to the effect that all I had to do was to press that one button (build?) to generate the WAR file. It took a long while to generate this War file. Intellij used an Ant file to build the war file. The team later migrated to Maven, but that's when my term was up.

Software development was very slow because I wanted to write code, compile and test code often in the web browser. There wasn't unit code testing.

So here's my question:

What does the industry do (for example FANG) where if I wanted to just modify ONE java file, how would I go about in a much faster fashion to compile that one file instead of recompiling every file to create a WAR file and then put it into the tomcat container and run it?

I just want to minimize the amount of time from inserting code to checking out how it behaves.

I tried to write my own batch script but taking into account all the thousands of java dependency was a nightmare. this this post. This is my first post since many years ago. I appreciate for someone to shed light on how I can approach this.

I wasn't happy with using this IDE solution because there was a very senior software engineer who didn't use an ide, isntead, used emacs on his mac. He got things done fast, but I didn't understand how he would go modify a few files and got it to run on tomcat. I can't imagine that he'd recompile everything. So please share with me some work flow using IDE and also preferably not to use an IDE.

I did my research and google but couldn't find an examnple tutorial of how such would be done. One reason I ask this question is because if I ever get a software engineering job, I would not want to modify one file and take forever to recompile everything.. it just doesn't sound like the way to go. This wasn't taught in school.

If there is a git project that deals with the above.. I'd be grateful if someone can send me an exampile git project that deals with the above webapp scenario.


I posted this question below but there appear to be dumb trolls who don't care to read the question:
https://softwareengineering.stackexchange.com/questions/395553/how-to-modify-and-compile-only-few-java-files-among-many-thousands-and-deploy?answertab=active#tab-top

Thank you. Gordon
 
author & internet detective
Posts: 39392
763
Eclipse IDE VI Editor Java
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Gordon,
Welcome to CodeRanch! You stumbled upon one of the limitations of Stack Overflow. It doesn't do so well at things that are discussion based or subjective like this. We on the other hand enjoy discussions too!

As you noticed, the javac command recompiles everything. On most large projects, the team uses a build tool. Maven and Gradle are most common these days. Older projects tend to  use Ant. All three of these have the ability to compile just the files that changed. That feature is called "incremental compilation." For example, you get incremental compilation in Maven unless you set an attribute to turn it off. I would save Maven and Gradle are industry standard. They also help with the dependency issue you mentioned.

Your IDE also does incremental compilation. (In fact, it also compiles often even before you save a file to let you know of errors.)
 
Saloon Keeper
Posts: 10415
223
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
In addition to what Jeanne said, if it takes a long long time to build your WAR, the application is probably not modular enough. Thousands of classes in a single assembly? Bad design.
 
Jeanne Boyarsky
author & internet detective
Posts: 39392
763
Eclipse IDE VI Editor Java
  • Likes 2
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:In addition to what Jeanne said, if it takes a long long time to build your WAR, the application is probably not modular enough. Thousands of classes in a single assembly? Bad design.


Ooh. Good point.

To spell that out a litter more since Gordon is a student:
IDeally the projecct would have one or more Java project it depends on. Then you need to just rebuild the jar it is in and not recompile all the code in other jars.
 
Rancher
Posts: 4177
47
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
A couple of things leapt out at me from that.

gordon prog wrote:
Software development was very slow because I wanted to write code, compile and test code often in the web browser. There wasn't unit code testing.



Dear lord.
At least youre not working there anymore.
That's a recipe for disaster.

gordon prog wrote:
I wasn't happy with using this IDE solution because there was a very senior software engineer who didn't use an ide, isntead, used emacs on his mac. He got things done fast, but I didn't understand how he would go modify a few files and got it to run on tomcat. I can't imagine that he'd recompile everything. So please share with me some work flow using IDE and also preferably not to use an IDE.



I don't really have anything against emacs, but it's not exactly a standard development environment for Java.
As others have said, the root problem is going to be the sheer monolithic nature of the project.

The one I'm on has a fair number of modules, and the build time across the module is 10-15 minutes, but that is mostly taken up with unit tests.

Since the IDE compiles the classes as I go along anyway, it is possible to simple lift the class(es) I've modified and drop it into the unpacked Tomcat deployment if I want to do a hacky test.
But note that that is a hacky test, and not something I would rely on to actually ensure things work properly.
 
Bartender
Posts: 20928
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jeanne Boyarsky wrote:

Stephan van Hulst wrote:In addition to what Jeanne said, if it takes a long long time to build your WAR, the application is probably not modular enough. Thousands of classes in a single assembly? Bad design.


Ooh. Good point.

To spell that out a litter more since Gordon is a student:
IDeally the projecct would have one or more Java project it depends on. Then you need to just rebuild the jar it is in and not recompile all the code in other jars.



Upvote.

I have used Emacs as an IDE for Java. There's a plugin for that. It's not as good as, say Eclipse, but at the time, I was editing on a 200MHz Pentium. Emacs is a monster, but not compared to most Java IDEs. Plus the machine in question wasn't running a windowing system.

I've never done an in-depth study, but my impression has always been that given a set of files, the javac compiler will recompile only the files that need it. That is, if you delete all the target classes, then obviously all the files must be re-compiled, but the process should be much faster if you leave the compiled classes in the compilation target class directory, since only the changed source files need full compilation. Even if this wasn't true, however, then even Emacs could run the java compiler as part of a file-save operation instead of waiting for a batch build command.

Easy way to test. Take a large Maven (or Ant project). Change 1 file, save it, do "mvn compile". Note timing. Now do "mvn clean compile". If all files are compiled all the time, then the two times will be about the same and fairly long. Otherwise the second timing will be much longer than the first. And no, I don't think Maven is where the decision to do selective compilation is made, so if there's a difference, it's almost certainly javac.
 
gordon provi
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Dave Tolls wrote:A couple of things leapt out at me from that.

gordon prog wrote:
Software development was very slow because I wanted to write code, compile and test code often in the web browser. There wasn't unit code testing.



Dear lord.
At least youre not working there anymore.
That's a recipe for disaster.

gordon prog wrote:
I wasn't happy with using this IDE solution because there was a very senior software engineer who didn't use an ide, isntead, used emacs on his mac. He got things done fast, but I didn't understand how he would go modify a few files and got it to run on tomcat. I can't imagine that he'd recompile everything. So please share with me some work flow using IDE and also preferably not to use an IDE.



I don't really have anything against emacs, but it's not exactly a standard development environment for Java.
As others have said, the root problem is going to be the sheer monolithic nature of the project.

The one I'm on has a fair number of modules, and the build time across the module is 10-15 minutes, but that is mostly taken up with unit tests.

Since the IDE compiles the classes as I go along anyway, it is possible to simple lift the class(es) I've modified and drop it into the unpacked Tomcat deployment if I want to do a hacky test.
But note that that is a hacky test, and not something I would rely on to actually ensure things work properly.




Dav or anybody else, for the sake of me learning this one little skill of working on a massive project, I'd like to learn how to setup my ide (eclipse?  I'm requesting the eclipse environment because that's free and it can do web development whereas the intellij community version doesn't allow web development.) with an open sourced project on github so that I can download it and play with it as though i'm facing my first day of real work.  Important:  i request that this be some sort of web application that uses tomcat.  I assume there's such a free project on github, if not, maybe start a small one and we build it for knowledge building sake.  I request the project also be Java based.  It would be really helpful if this thread / youtube video can walk me through the setup.  I can't find such tutorial and I think it would help a ton others.  Also, the project would have unit testing etc.  I know I'm asking a lot but I think it would help a lot of newbies out there.  I've read books before, but it doesn't get at what people face in real world and put it all together.  I'm at a point that I feel pointless to prepare for job interviews when I feel that I don't know enough where I'd be kicked out the first day.  Thus this request.  For peets sake, for sure, this request would be shot down at StackOverfool.

I really wish someone can tutor me over this "getting started the right way hurdle."  School didn't teach it.  Landed my internship, didn't teach it, and I suffered.  I'm also interested in the dropping the jar file .. I'm not sure how to do the "unpacked Tomcat deployment" by dropping in a single jar file and have tomcat auto update.  Historically, tomcat was slow to auto update.  I dont' recall, since it's many years back so I don't recall how it was done.. he had me somehow compile for this one jar file and copy it into one of tomcat's directory, and restart tomcat (without recompiling to a war file) because it would not immediately recognize the change jar, but that lead to out of memory errors on tomcat.  

I suppose what I'm asking for is ... is there any chance someone can create a youtube video on the above? =)  It's much clearer than to type and it's much easier to follow along with fewer errors.  And to encourage you... maybe you can  monitize on the video too! =)  it's a win - win right?

To answer people's question:  reason why the company made one big fat war file was because there's a lot of web apps that the (company / school) hosted, and so each war file belong to one web app.  i suppose it was an easier way to keep track of web apps.   The war file was a bit over 100 megabytes.

Thanks a bundle,
Gordon
 
gordon provi
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Stephan van Hulst wrote:In addition to what Jeanne said, if it takes a long long time to build your WAR, the application is probably not modular enough. Thousands of classes in a single assembly? Bad design.



How would you do it?  I always thought one web app would be one war file, which can have many jar files and .class files.  Are you saying that one big fat web app would have many war files?
 
gordon provi
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Tim Holloway wrote:

Jeanne Boyarsky wrote:

Stephan van Hulst wrote:In addition to what Jeanne said, if it takes a long long time to build your WAR, the application is probably not modular enough. Thousands of classes in a single assembly? Bad design.


Ooh. Good point.

To spell that out a litter more since Gordon is a student:
IDeally the projecct would have one or more Java project it depends on. Then you need to just rebuild the jar it is in and not recompile all the code in other jars.



Upvote.

I have used Emacs as an IDE for Java. There's a plugin for that. It's not as good as, say Eclipse, but at the time, I was editing on a 200MHz Pentium. Emacs is a monster, but not compared to most Java IDEs. Plus the machine in question wasn't running a windowing system.

I've never done an in-depth study, but my impression has always been that given a set of files, the javac compiler will recompile only the files that need it. That is, if you delete all the target classes, then obviously all the files must be re-compiled, but the process should be much faster if you leave the compiled classes in the compilation target class directory, since only the changed source files need full compilation. Even if this wasn't true, however, then even Emacs could run the java compiler as part of a file-save operation instead of waiting for a batch build command.

Easy way to test. Take a large Maven (or Ant project). Change 1 file, save it, do "mvn compile". Note timing. Now do "mvn clean compile". If all files are compiled all the time, then the two times will be about the same and fairly long. Otherwise the second timing will be much longer than the first. And no, I don't think Maven is where the decision to do selective compilation is made, so if there's a difference, it's almost certainly javac.



-------------

I was a newbie, so all I knew at the time was to press the one button in intelij and I think it was called compile or something and it would go and build the whole war file.  intellij kicked off some sort of ant script to do the war compilation.  it would take a long time like x minutes or so to recompile even if I change just a single jar file.  I think I had click on the clean button too before each recompilation so that there's no error in the ant script, so that's one of the problem.
 
Greenhorn
Posts: 25
1
MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Eclipse for JavaEE Developers includes JUnit testing. This is probably the version you want to download anyway if you do web applications. Maven and Github support should be included as well.
As for recommendations for Youtube tutorials you would have to post what framework you are planning to use or has been used in the existing project. Spring MVC or Spring RestAPI projects are very different than projects with other frameworks, but they are all Java frameworks.
If you want to improve Modularity in your project I suggest you look for "Java Microservices" on Youtube. If you look for Java Modularity you end up with lessons about Java9+ in the 'SE' world and not so much in the 'EE'/'Jakarta' world.
 
gordon provi
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jan Daermann wrote:Eclipse for JavaEE Developers includes JUnit testing. This is probably the version you want to download anyway if you do web applications. Maven and Github support should be included as well.
As for recommendations for Youtube tutorials you would have to post what framework you are planning to use or has been used in the existing project. Spring MVC or Spring RestAPI projects are very different than projects with other frameworks, but they are all Java frameworks.
If you want to improve Modularity in your project I suggest you look for "Java Microservices" on Youtube. If you look for Java Modularity you end up with lessons about Java9+ in the 'SE' world and not so much in the 'EE'/'Jakarta' world.



I've been away from programming for 3 years.  I didn't know Spring.  But I would like to use a project with Maven.  I've heard that React is popular.  I don't want to waste a lot of people's time so I'm thinking, if you have an open source Spring project or whatever framework that uses Eclipse, and have Junit testing, and are willing to make the video that would do it for me.  Most important for me though is:   I'd like to just change one java file of the many and have eclipse recompile only that one file and not anything else =)  this project of course would use Tomcat, so it's an webapp.  These are the only requirement.. frameworks, etc, it doesn't matter, and I can learn something on the side too=)   Well, I'm fine with any framework just to minimize the burden and get this tutorial off the ground; besides, if and when I get a new job, I don't get to call the shot as to what to framework to use, so all is fair game at this point. =)  I'm just pretty desparate to see this fast compilation / work flow.  This is the whole intent of my post.  I just want a tutorial on how it's really done in the real world so that I have some confidence that I don't have the "imposter syndrome." =)  I suppose once the tutorial gets posted using Eclipse, if people who use emacs can chime in how they go about doing things of just compiling that one file, that would be fine too.  I'm curious about this emacs or simple editor thing because this particular senior architecture guy was able to churn out code FAST.  I don't know what his setup is or how he's able to code without an ide.  he used a mac and was basically on the command prompt.  Some say it's best not to code with an ide because of the setup issues one may run into going into a new job.  Thus, I'd like to see things done with and without an ide, but the main thing is, i'd like to see change in one one java file =)  without recompiling everything else.

Thank you,
Gordon
 
Jan Daermann
Greenhorn
Posts: 25
1
MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
1. I would not worry about the compile time issue, that’s a non issue if you start your own project.
2. If you are worried about the relevance of the tools for your job search, go on stackoverflow in the job section and check what is required. I would think that many jobs require experience with an IDE environment, a backend environment (Java Spring 5.0 or Java Spring Boot 2.0, very similar) and a front end environment (html, css, Javascript plus JS libraries like React or Angular).
3. There are tutorials available: udemy offers a course “fullstack development with Spring Boot 2 and React” for $12.- in the moment. There is also a good book with the same name on Amazon that gives a fun overview without going into too much detail.
4. Just to make sure we are on the same page: Backend means any Java framework (like Spring Boot 2.0), typically edited on an IDE like eclipse. It handles all the data persistence (JPA, Hibernate), security, backend logic and deals with requests that reach the server. It runs on a separate server and typically has access to a database server as well. Nowadays it is often a RESTServer or RESTApi -> it does not return a webpage anymore, it returns a dataset (JSON object, representational state transfer). Frontend often means html, css, Javascript sites, often edited in a different IDE (Visual Studio Code) that typically runs in conjunction with a JS development tool like “npm” or “yarn” (The JS world is a little more complicated here). React and Angular are the graphics libraries for JS that make things easier. npm includes its own webserver that runs the websites simultanously, so you see changes in your code on the flight. The frontend consumes the JSON object coming from the backend and display it. After you have digested this jungle you are a fullstack developer .
 
gordon provi
Greenhorn
Posts: 6
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

Jan Daermann wrote:1. I would not worry about the compile time issue, that’s a non issue if you start your own project.



This is at the crux of the problem, that once I get a job, it's no longer my project that I start.  I have to be able to quickly modify code and compile just a few java files to test things out.  This is the whole point of this thread.


Jan Daermann wrote:
2. If you are worried about the relevance of the tools for your job search, go on stackoverflow in the job section and check what is required. I would think that many jobs require experience with an IDE environment, a backend environment (Java Spring 5.0 or Java Spring Boot 2.0, very similar) and a front end environment (html, css, Javascript plus JS libraries like React or Angular).
3. There are tutorials available: udemy offers a course “fullstack development with Spring Boot 2 and React” for $12.- in the moment. There is also a good book with the same name on Amazon that gives a fun overview without going into too much detail.
4. Just to make sure we are on the same page: Backend means any Java framework (like Spring Boot 2.0), typically edited on an IDE like eclipse. It handles all the data persistence (JPA, Hibernate), security, backend logic and deals with requests that reach the server. It runs on a separate server and typically has access to a database server as well. Nowadays it is often a RESTServer or RESTApi -> it does not return a webpage anymore, it returns a dataset (JSON object, representational state transfer). Frontend often means html, css, Javascript sites, often edited in a different IDE (Visual Studio Code) that typically runs in conjunction with a JS development tool like “npm” or “yarn” (The JS world is a little more complicated here). React and Angular are the graphics libraries for JS that make things easier. npm includes its own webserver that runs the websites simultanously, so you see changes in your code on the flight. The frontend consumes the JSON object coming from the backend and display it. After you have digested this jungle you are a fullstack developer .




These do not teach people how to modify one or few java files in a major web app program.  Thus point of this thread.
 
Jan Daermann
Greenhorn
Posts: 25
1
MySQL Database Spring Tomcat Server
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Sorry for the misunderstanding.
 
Dave Tolls
Rancher
Posts: 4177
47
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That's what unit tests, and smaller scale testing of concepts are for, really.

You should already have a fairly solid fix before deploying most of the time, by which point you probably really do want to do a full build to ensure all existing tests pass (no point deploying a fix that doesn't pass your test suite) and there are no side effects.

Believe me, if you're that new to the whole thing I really wouldn't sweat it.
Your CV will make it pretty clear you're coming in at the ground level.

One thing I will say, getting a git hub project may give you some insight into how that project works (and possibly how, eg, a maven project builds in general), but it won't cover you for any one particular job.  I contract.  Pretty much every one has a different set up.  Turn up on the first day and hope the people you are working with are at least vaguely organised.

As for unpacking your war files in Tomcat, there's a setting in server.xml, under the host section called unpackWARs.  Set that to true and when tomcat deploys a WAR file it will unpack it into the webapp directory first.
This gives you an unpacked application where you can make minor changes without too much trouble (like dropping in a new jar file, or modifying a bit of HTML/CSS/Javascript).

The section you're looking for looks something like:

<Host name="<host name>"  appBase="webapps"  unpackWARs="true" autoDeploy="true">


If you deploy a new WAR file, remember that this will blat your current unpacked one when it is deployed.

Bit rambly there, but hopefully something of use...
 
Tim Holloway
Bartender
Posts: 20928
127
Android Eclipse IDE Tomcat Server Redhat Java Linux
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator

gordon provi wrote:
This is at the crux of the problem, that once I get a job, it's no longer my project that I start.  I have to be able to quickly modify code and compile just a few java files to test things out.  This is the whole point of this thread.


And you'll be subject to the same limitations as everyone else who ever worked on the project, so unless you mess something up, you can't be faulted for taking any longer than anyone else.

And even my largest Java projects build MUCH more rapidly than my C/C++ projects did. I used to have to find all sorts of ways to amuse myself while a C build was done. Even on today's processors, a build of the Open-Source FreeCAD program takes something like an hour but I hardly have time to get coffee before a java build is done, even from scratch.

Unit tests are important, but since they add even more classes to the compile mix, they won't gain you speed in that sense. On the other hand, unit tests are generally designed so that you don't need to fire up a big complex application framework to run them. And cranking up a copy of WebSphere just to test a 1-line change can get old real fast.

Tomcat, incidentally is shipped with the WAR unpacking feature switched on by default. And you can save a lot of time by setting up Tomcat to use the unpacked WAR that you are (presumably) assembling in your project directory (Maven projects, for example, assemble WARs in their "target" directory). Be aware, however, that this approach has pitfalls, since some applications do require undeploying and redeploying the app, and sometimes Tomcat as well. Changes to static content (javascript, CSS, images and so forth) are OK. JSPs should recompile. Static classes are the biggest problem, since without redeployment, stuff gets out of sync. Also, Tomcat does not adopt changes instantly. It has a watchdog class that scans periodically for changes to the WAR and will not apply the changes until the next scan. So there may be some delay.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!