Does erroneous variable overwriting indicate a memory overflow?

[Chris Crawford]

First off, I'm not a real programmer, although I've been writing code for over fifty years. For me, programming is a means, not an end, so I minimize my knowledge to what is necessary to get the job done -- and usually the programming is not the primary effort in my work. I explain this to caution you that I really don't know that much about Java, even though I've been using Java for nearly 20 years and I have written tens of thousands of lines of code in Java.

So here's my problem: my code is illegally overwriting a variable. Here are four lines of code, with their output:

System.out.println(leftButton.getText());
centerSpinner.setValue(myReaction.getConstantCenter());
System.out.println(myReaction.getConstantCenter());
System.out.println(leftButton.getText());

pBad_Good
0.5
0.5

Thus, writing a value to the variable centerSpinner overwrites the value of the variable leftButton -- which has nothing whatsoever to do with centerSpinner! These two variables are both class-level variables in the same class.

This is not a huge complicated program; it doesn't have any big data structures. No images, no big buffers. The data structures it uses fit into an 86 KB XML file. The problem hits right at the launch of the program, so it's not some sort of memory leak.

I noted that the memory usage of Eclipse when this happened was 1.03 GB, which leads me to suspect that Eclipse was working with a 1.00 GB memory allocation and overran. But shouldn't I have gotten an OutOfMemoryError?  I ran another program I'm working on, and it worked fine even though Eclipse was using 1.06 GB of memory.

I spent some time looking through discussions here on CodeRanch about OutOfMemoryErrors and garbage collection, which convinced me that it will cost me several days to learn enough about this to work this out myself.

I'm not asking anybody to solve my problem for me; what I'm asking is whether this problem looks like excessive consumption of memory despite the contraindications.

[Campbell Ritchie]

Well, well, well, well, well....
No, it isn't a memory overflow problem. You don't need to know about memory and garbage collection to work out that behaviour. But you do need more information.
Please remove some of the printXXX() instructions to verify that you are really printing what you think you are printing. Also please print the two variables System.out.println(lefftButton); and similarly for the Spinner.
Please go through the code and look at wherever you are assigning to either of those variables: centerSpinner = xxxxx; and  leftButton = xxxxx;.

[Chris Crawford]

I think you nailed it. This old dinosaur is still locked into the old procedural model of code execution, and it never occurred to me that various Listeners might intervene during initialization, but in fact that's what happened. Initializing a spinner triggered its ActionListener, which in turn caused it to modify leftButton. I was sure that it had to happen during the code that I copied here, but sneaky old Java pulled that trick under the table.

Is there a standard way to prevent Listeners from activating during initialization? I had experienced a similar problem with another program some years ago and solved it by setting up a global boolean that blocked action by Listeners during initialization. At the outset of initialization, isInitializing = true. Every Listener had as its first statement an if-statement checking the value of isInitializing. At the end of initialization, isInitializing is set to false. It's my kind of programming solution: crude, inefficient, but effective. There HAS to be a better way. Is there?

[Paul Clapham]

Chris Crawford wrote:Is there a standard way to prevent Listeners from activating during initialization? I had experienced a similar problem with another program some years ago and solved it by setting up a global boolean that blocked action by Listeners during initialization.

The global-boolean idea is fine, I notice that I have done that in the past. But you could try not adding the Listeners until the component is ready to listen properly, I suppose.

[Campbell Ritchie]

I would prefer boolean completelyInitialised = false;. That avoids using a double‑negative.
But I am a bit surprised that listener is jumping the gun like that. Please show us how you are adding it and how you are making the GUI visible. I notice that JSpinner doesn't support action listeners; you would use a change listener or similar. A JButton does however support action listeners.

[Chris Crawford]

Thanks again for the suggestions. I have indeed solved the problem by assiduously implementing the isCompletelyInitialised variable throughout every nook and cranny of the code. I am now advancing to other, even dumber mistakes. Onward!

[Campbell Ritchie]

Chris Crawford wrote:Thanks again . . .

That's a pleasure but we would still like to see fo ourselves what the code looks like that is going wrong.

[Stephan van Hulst]

Paul Clapham wrote:The global-boolean idea is fine

I strongly disagree. Global variables are never fine.

But you could try not adding the Listeners until the component is ready to listen properly, I suppose.

This is the real solution. Initialize and layout all components, and add listeners only at the very end.

[Campbell Ritchie]

Or there may be something wrong with the way the listener is added, so it is triggered prematurely.

[Chris Crawford]

...but we would still like to see fo ourselves what the code looks like that is going wrong.

There are two overall processes that cause trouble. The first is program initialization during which I'm setting up UI elements and their listeners. This wasn't difficult to handle. The real difficulty comes with the second step, during which I am loading data from a file. This data must be stuffed into the various control devices, which in turn triggers their listeners. A file can be loaded at any time during program execution, and because control devices influence each other (storing one kind of data into button A can enable or disable spinner B, for example), I don't want those devices triggering events while I'm loading data from the file. This is why I prefer to just disable all listeners during initialization and file loading.

It's true that a careful analysis of the sequence of initialization and file loading would enable me to optimize the sequence so that improper events wouldn't be triggered. But the hours I spend on that improvement would not be usable for the more important design issues I'm struggling with.

As to global variables, I grew up on FORTRAN and assembler, where everything was global! I've learned enough to greatly reduce my use of globals, but there remain a few areas that are troublesome. In particular, because their scope is not the same as the class they're working within, event handlers are particularly difficult for me, so I end up falling back on globals occasionally.

[Campbell Ritchie]

Thank you. I still can't see how you are firing a listener by reading a file, however.

[Chris Crawford]

It's not reading the file that triggers the listener; it's stuffing its values into the various UI devices. The program is an editor. The user loads a file. The file contains numbers and strings that the user can edit. The program loads the file and puts some of the numbers and strings into text edit boxes, spinners, buttons, and sliders so that they can be edited. The act of putting these numbers and strings into various devices triggers a ChangeEvent. The processing of the change event was causing other undesirable effects.

[Jesse Silverman]

So weird to read the words that you are "not a real programmer" tho you explain quite clearly what you mean.
I remember many people I trusted that definitely considered you on the short list for "Great Atari Programmers", at least in terms of results.

I had just happened to have  re-read this as bedtime reading recently back the last time you had popped in here (pages 3 thru 7 most relevant):
http://www.cgwmuseum.org/galleries/issues/cgw_1.1.pdf
At least BASIC died since then...

I didn't answer the question you posed either to the thread or just one poster, the last time you were here, but yes, even tho I forgot a lot of them and they probably wouldn't come back for an hour or two, that A9 is the opcode for LDA #immediate is burned into my brain more than all the others.  One of my dearly departed programming mentors described the 6502 as "something that would need a lot added to the instruction set to be RISC" humorously but seriously, he preferred the Z80...we worked on tens of millions of lines of mostly C code on x86/x64 together, and often looked at dumps together with fiendishly obfuscated call stacks...

[Paul Clapham]

Stephan van Hulst wrote:

Paul Clapham wrote:The global-boolean idea is fine

I strongly disagree. Global variables are never fine.

So I went back to the code where I used this solution (quite a few years ago). The boolean variable is a private member of the JPanel class which has a lot of data items. So it's not a "global" variable. It's part of the panel's state ("Are the panel's components being loaded?"), which other code in the panel can react to as necessary.

But could I get rid of it by moving the code which adds listeners so that it runs after the components are loaded, rather than immediately after each of them are created? It turns out, not really. One of the components of the panel is a JList which, when selected from, fills in new values in the other panel. So it's not just an issue when the panel is first instantiated. Maybe I could have that process remove all of the listeners, then fill in the new values, then add all of the listeners back? Maybe...

So what's the big deal about the listeners anyway? The panel has undo/redo built into most of its components, so when the user sets the value of a component, a listener notices that and writes an edit to the undo manager. But when the program sets the value, the resulting change should be ignored by the undo manager. Problem is, the undo manager can't tell whether the user or the program set the value of a component. That may not be Chris's problem but it's the same sort of problem.

And likewise when the user clicks the "Undo" or the "Redo" button, the resulting change to the component shouldn't result in writing a new edit to the undo manager. Therefore there's special code to tell the undo manager "Hey, it's me, that wasn't the user making that change, just ignore it."

So in my case it's the undo/redo feature which made me do that. Would it be better for the various panels which use the undo manager to be able to tell it to stop listening to change reports, and then later to tell it to start listening again? Probably. Instead of code all over the class having to ask whether a panel-loading event or an undo/redo event is happening and behave accordingly, I could just have the code which handles those events advise the undo manager of them. I might look into that.

[Chris Crawford]

I remember many people I trusted that definitely considered you on the short list for "Great Atari Programmers", at least in terms of results.  

Well, yes, I think I can claim to have mastered everything about the Atari HCS. But that was a simple system. The 6502 was a beautifully clean CPU with a delightful instruction set. The HCS did have a number of tricky features, most notably the Display List Interrupt. However, I was 30 years old and full of the young man's cognitive intensity. I could keep all the variables in a big assembly language program in my head. Nowadays it's all I can do to juggle just one ball...

Nowadays, languages like Java are immensely more complicated than the 8-bit machines we played with forty years ago. Does anybody believe that there is a person on this planet who has mastered every single Class and Interface in the standard set? When was the last time you used the FormatFlagsConversionMismatchException class? Or the InternalFrameFocusTraversalPolicy class? I'll bet that, buried somewhere deep in there is a BalanceTheFederalBudget Class and an ImplementWorldPeace Interface.

We were able to put together everything that you could know about the HCS in a technical document less than 200 pages long. By contrast, there are hefty books just explaining Swing. The fact that there are so many sites on the web (of which this one is arguably the best) devoted to Java gives an idea of just how immense this technology is. I just checked and there are 389 THOUSAND topics in the "Beginning Java" section of the forum, and 290 thousand topics in the "Java in General" section. I doubt that there were as many as 10 thousand programmers using the Atari back then.

The Atari Pond was so small that it was easy to be the biggest fish in the pond back then. Nowadays, though, the Java Ocean is so big that I feel like a bacterium floating in it. I have enormous respect for the people who have devoted their time to learning a big hunk of this monster. I'm too busy with other things: designing interactive storytelling systems, teaching, fixing my tractor, reforesting my land... sheesh. So thanks again to everybody for the help!

[Jesse Silverman]

That captures so eloquently the biggest challenge I have.

No individual technical or design bit eludes my understanding when I seriously try to comprehend it, I think I have the soft skills to work productively with others....

but the awesome scope of Everything, for someone who like us, was used to "knowing everything" about a system, or knowing quite a good deal about it and knowing that somewhat better people truly knew it all...is psychologically daunting.

Stuck in a strict mindset of this nature, one would drown in Impostor Syndrome, even people that know much more than most, and are more productive than most.

What you said is multiplied much worse if one is hopping from Python to Java to .Net to C++ across different hardware and Operating Systems.

I was suitably impressed when developers got their text adventures running in ~16k on a bunch of cassette-based machines with wildly different processors, sometimes by themselves...

I didn't think so many jobs would still feel like that so much later, (on top of very broad and deep 'stacks' that nobody could possibly fully know) but they do...

That certainly also shows one of the core values that continues to bring people here -- when there is so much stuff to know that it would be impossible to try to know everything, how do you know what to learn first and best?  The certification exams provide some guidance, and people pointed in that direction gravitate here.

Cheers.

[Jesse Silverman]

There was a second related point that you actually addressed in there, but not part about the programmers.

There's some quote going around that "Developers are good because of what they can figure out, not because of what they know..."

Your 1981 article talked about people not appreciating the difference for operating on partial or missing information in a wargame, because they were used to the artificial situation of "knowing everything" when they played because they had to because they were simultaneously running the game and playing...

That is the skill I am trying to calmly adapt to in programming -- not only do I not know everything, those adapting more successfully don't know everything either....for most developers, everything changed from "good developers know everything about the language/environment they are operating in" to "good developers don't freak out when they don't know everything about the language/environment they are operating in, which is all the time"...you can learn more, and you will, but you will find a way to be productive the whole time you are climbing that mountain, which will be forever.

In the past I not only got by but excelled not because I knew "everything" but more than most of the Senior Developers (not about the business logic, but about the languages and environment)...I could pretty reliably get to the root cause of problems that were totally befuddling senior developers trying to lead a team adding new business logic without breaking anything that needed to keep working...sometimes they had spent two days getting nowhere and I would find it in one day, other times it would take me two or three (tens of millions of lines of code with mixed languages together in the same dang call stack) but they couldn't afford a third day.  It was nuts but paid the bills.

Python was really simple 25 years ago, not so much now.  Almost everything else is more complex than that....I am having adventures in C++ right now.
What got added/changed in C++ 11 from C++ 03 is 50 times as much as there was for anyone to ever know about Modula 3 or C for that matter, to C++ 14, C++ 17 C++ 20 maybe less than that jump but still so much.

You can (almost) write C90 code and it will "work" but (almost literally) nobody considers that decent C++ coding...a lot of catching up involves learning "new basics" from scratch....the term C/C++ is so meaningless it should return NaN for a result.

[Tim Holloway]

Going back to the original question, some might have erroneously interpreted that to mean "memory overwriting". Memory overwriting isn't supposed to happen in Java, based on the fundamental design specs of the language. The JVM meticulously bounds each Object and primitive in isolated (and not necessarily fixed or even contiguous) locations. It works with abstractions, not memory. Memory overwriting is a problem with languages like C and C++, and one which Rust attempts to address at that level. JVMs don't permit raw memory access and every read and write to object/primitive memory is tested to make sure you don't get things like buffer overflows.

[Chris Crawford]

Mr. Silverman, I wonder how much of the huge change in the programming culture is cause and what is effect. Are programming languages immensely more complicated because the Internet makes it possible to get help from others? Most people end up learning about computers from their peers -- has the Internet vastly expanded the set of peers, thereby vastly expanding the complexities of computer systems? If so, I think that we'll eventually have to dump the charity approach used in places like this website. There will be more and more people needing more and more help with more and more complicated technologies, and we'll need full-time helpers to make all this possible -- which in turn means that we need a model that pays them for their efforts.

I just now noticed the PIE system on this website, and bought some pie. Lord knows I owe you guys a lot. How well has the PIE system worked out? Is the end result fair to the contributors?

[Tim Holloway]

Thanks, Chris!

As far as the idea that places like the Ranch will someday become obsolete, let me relate some history.

Back when I started on mainframes, languages, language environments, operating systems and peripherals were all fairly simple. My first language was Fortran and supposedly there were only 10 unique statement types in the language, so learning the language itself was simple.

But Fortran was very limited. It had no built-in character or string support, so code I wrote for the school mainframe often required considerable modification to run on our mini-computer. And vice versa.

This was also before language support for Structured or Object-Oriented programming. Large programs were often full of "spaghetti code". We also had limited RAM and persistent storage capabilities, so there had to be limits to our ambitions.

Things got a lot more interesting when GUI systems like the Macintosh, MS-Windows and others hit the streets. You had a whole raft of graphics and widget functions that you had to be able to use. Plus, a well-designed GUI app has a LOT of interconnections - an operation like File Save might have a menu option. a toolbar button, a right-mouse (context menu) click, keyboard shortcut and more and all of them required a lot of manual wiring as a general rule. It was a rare GUI app that weighed less than a Megabyte.

Then the Internet came in and databases. Suddenly apps needed to connect to 18 different social systems, deal with emails, workflows and schedulers, handle esoteric data formats like XML, JSON and YAML, etc., etc., etc.

So it's a LOT more complicated. As an OS-level mainframe progammer we had a rack about 10 feet long full of IBM manuals that we regularly referenced and frequently updated (IBM used a loose-leaf format to avoid having to send out entire new copies). I don't even want to THINK about how long a rack full of manuals for even the core Java language environment would be at this point if it were on dead trees. Back when I still printed, Sun routinely created 400+ page documents. That's before we add in the third-party stuff like Spring Framework.

Heck, even the size of Python's documentation is intimidating.

So does this mean that only full-time paid company personnel can support all that? Dream on!

In actual point of fact, corporate product support has dwindled over the years. In ancient time, IBM might actually have an office right next to our datacenter. By about 1985, it was down to dial-up "Please Stay on the Line, Your call is very important to us!". Eventually it deteriorated into the stereotypical "monkey-with-a-script" and "Have you tried turning it off and back on again?" Even the largest and best-known corporations have mostly ended up using forum software just like the Ranch. And often the corporate responders are not the most knowledgeable people on those forums.

It's one of the biggest reasons why I prefer open-source these days. Free software often gets better support from free forums than the expensive stuff gets from corporate contacts.

So expect to see us around - and handing out information - unpaid - for the forseeable future.

[Campbell Ritchie]

Tim Holloway wrote:. . . handing out information - unpaid - for the forseeable future.

That means forever.

[Campbell Ritchie]

Good grief! CC has given me some pie. Thank you