This week's book giveaway is in the Kotlin forum.
We're giving away four copies of Kotlin in Action and have Dmitry Jemerov & Svetlana Isakova on-line!
See this thread for details.
Win a copy of Kotlin in Action this week in the Kotlin forum!
  • Post Reply Bookmark Topic Watch Topic
  • New Topic

Type Safety in Derived Classes  RSS feed

 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
My recent discussion here about multiple inheritance got me looking into some other ways to avoid unsafe types being passed to routines that don't themselves care about the types of their parameters, but that, for business reasons, shouldn't be passed pairs of things unless the pair is on a "safe" list.

Here's an example:

My top-level class is called Person. It is package-private and can store a reference to an instance of itself in spouse:



There are four public subclasses of Person, each called some variation of Parent[12XY]:



In another package, each of those classes has one child class:



An instance of a class that is a child of Parent1 or Parent2 may "marry" (be paired with) an instance of a class that is a child of ParentX or ParentY. Instances of child classes of Parent1 or Parent2 may not "marry" each other, nor may instances of child classes of ParentX or ParentY.

To enforce this, I have created a static method in class KimDavis that will accept any two instances of Person, and type-check them at run-time:



That's nice and tidy (if you don't mind that lump of logic at Lines 7 through 10), but has two drawbacks: first, it only detects disallowed pairings at run-time; second, it provokes my IDE into giving me the dreaded "Exporting public type through non-public API" warning.

To enforce the rules on who can marry whom, I have replaced that one method with eight overloaded methods of the same name, one for each of the allowed types of marriage:



This does what I want, enforces type-safety at compile time, and no longer provokes any ominous warnings from my IDE. However, it means I have had to manually figure out all the combinations of possible marriages and code a method signature for each. In any slightly more complicated scenario, the chances of missing one (or allowing one that should be prohibited) would be significant (if you're me they would, anyway).

Is this a legitimate way to enforce a pair-wise restriction on classes, or am I missing something better here? (Or a problem with this approach altogether?)
 
Ron McLeod
Saloon Keeper
Posts: 1600
232
Android Angular Framework Eclipse IDE Java Linux MySQL Database Redhat TypeScript
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Although I have never tried this myself, one option would be to use compile-time annotation processing.
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:Is this a legitimate way to enforce a pair-wise restriction on classes, or am I missing something better here? (Or a problem with this approach altogether?)

My suspicion is the latter, because I think this is a WhatNotHow problem.

It's a bit difficult to answer for such abstract clasees, but in real life what you would need to do is answer the following question:
What is it that allows two people to marry? - or, if you prefer - What prevents one Person from marrying another?

If it has to do with some attribute (like for example, sex, if you live in a more orthodox country) then I suspect that enforcement by type is not what you want. If however, it genuinely is something to do with a "type" that cuts across an existing hierarchy, then one possibility would be to use a marker interface.

A good example of that is the RandomAccess interface, which basically says: "this List's get(int) method works in constant time".
So it's a bit like a "strategy" indicator; and ArrayList implements it, while LinkedList doesn't.

However, they should be used very sparingly. They can be easily overlooked, and I've seen several custom List implementations that should implement RandomAccess, but don't (I've written a few myself).

Perhaps if you could come up with a real-life example it might be a bit clearer, but I doubt that there's a single "magic bullet" answer to this question.

HIH

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:If however, it genuinely is something to do with a "type" that cuts across an existing hierarchy, then one possibility would be to use a marker interface.

I thought about that, but I am also aware of the general skepticism that accompanies their use, and it struck me that it imposes an obligation on programmers who will be calling the marry method to add the markers, whereas, if they are subclassing the various Parent* classes to create the instances they are passing to marry(), each of those subclasses is already "marked" with the class of its superclass (that is, the programmer can't forget to mark a class as being IS-A member of its superclass). Ideally, I'd like to be able to declare a method with a signature something like this:


Although, even that would require that it be coded twice, to allow for the converse case:



However, they should be used very sparingly.

So I've heard.


Perhaps if you could come up with a real-life example it might be a bit clearer, but I doubt that there's a single "magic bullet" answer to this question.

I'll probably regret this (looking at you, Stephan van Hulst ), but here's what I'm doing: I need to create a doubly linked list of three classes of object, called Source, Sink, and Filter. All three are subclasses of AbstractFilter, and, in practice, the actual instances I'll be dealing with will be subclasses of those three classes (so, in my original example, the part of AbstractFilter was played by Person, the parts of Source, Sink, and Filter were played by Parent1, Parent2, ParentX, and ParentY, and the instances I'll actually be dealing with were played by ChildAlpha, etc.). The rules are that the following pairs are allowed for connection (what the marry() method does in my example):

(Source, Sink)
(Source, Filter)
(Filter, Filter)
(Filter, Sink)

The order within each pair is significant, meaning that, for example, (Sink, Source) is not allowed. Source, Sink, and Filter are all public classes, but their superclass, AbstractFilter, is package-private. A static method in another public class can connect any two AbstractFilter objects. The goal is to enforce the rule that only pairs on that list of four, above, can actually be passed to that static method. Hence, my code looks something like this:


Using the above permits this:



Now, that does what I want, but it requires a list of overloaded connect() methods as long as my list of allowed pairs (which is rather longer than four, in my actual application, but that's my problem).

I like this approach, because the programmer calling connect() neither knows nor cares that there are a gazillion overloaded versions of it. The programmer only knows that, if they try to connect two instances that cannot be connected, they are prevented from doing so at compile time. It also allows me to add or remove pairs to/from the list without having to do more than add/remove another overloaded version of connect(). Pretty clean, once the initial set of overloaded methods is created. In my actual application, there are 18 of them, and that simply grew long enough that I began to doubt myself. Whenver I doubt myself, I come here, typically to experience the cleansing humility of discovering that my doubts were warranted, by learning that there was a better way all along. In this case, I'm thinking this way may, indeed, by acceptable. But I'm never too proud to learn alternatives from more experienced minds.

HIH

Buddy, you always do!
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:...it struck me that it imposes an obligation on programmers who will be calling the marry method to add the markers, whereas, if they are subclassing the various Parent* classes to create the instances they are passing to marry(), each of those subclasses is already "marked" with the class of its superclass (that is, the programmer can't forget to mark a class as being IS-A member of its superclass).


Oh, hey, that was a really dumb thing to say, wasn't it? I can apply the marker interfaces to my Parent classes, and those interfaces will be inherited by the Parents' subclasses.

That might be the ticket!
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Likes 1
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:I thought about that, but I am also aware of the general skepticism that accompanies their use, and it struck me that it imposes an obligation on programmers who will be calling the marry method to add the markers, whereas, if they are subclassing the various Parent* classes to create the instances they are passing to marry(), each of those subclasses is already "marked" with the class of its superclass (that is, the programmer can't forget to mark a class as being IS-A member of its superclass).

Hmmm. I think that's a documentation issue as much as anything. IMO, the main problem with RandomAccess is that it isn't highlighted enough in the List docs.

Take your "real-life" example: With two marker interfaces you could implement all your logic without resorting to overloading, viz:Now it's by no means the only way to do it, but I think (?) it solves your problem.

A disadvantage is that it's a bit more "cryptic" than overloading, so to save yourself writing four methods it's probably overkill; but don't forget that if you add 3 new types later on that number could increase substantially, whereas with the solution above the only thing a developer needs to remember is to have those new types implement the relevant marker interface(s).

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:Now it's by no means the only way to do it, but I think (?) it solves your problem.

It does. Winston, you're a genius. (But you knew that, right? )
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:It does. Winston, you're a genius. (But you knew that, right? )

Of course. Think nothing of it old chap.

Another thought struck me after I posted: rather than have those markers as separate interfaces, you could make them part of Connector - because that is, after all, the context in which they're used. Viz:and then just change the other classes accordingly.

Good luck.

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
That works. Now, I actually don't want those interfaces to be visible outside the package where AbstractFilter is defined (no one else would ever need them, and I don't want anyone tricking my connect() method into accepting something not derived from AbstractFilter). So, I made them package-private. This, alas, provokes one of those, "exporting not public type through public API" warnings at Line 31 (of your first code listing). This is a case where I think I can live with that, however. Also, before doing the actual connection, I need to cast from and to to type AbstractFilter, as that's the class with the methods needed to build the list. Heh, I suppose I could put the method signatures into ConnectTo and ConnectFrom, couldn't I? Then I could skip the cast and also stop feeling guilty about using marker interfaces!
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:That works. Now, I actually don't want those interfaces to be visible outside the package where AbstractFilter is defined (no one else would ever need them, and I don't want anyone tricking my connect() method into accepting something not derived from AbstractFilter). So, I made them package-private. This, alas, provokes one of those, "exporting not public type through public API" warnings at Line 31 (of your first code listing). This is a case where I think I can live with that, however. Also, before doing the actual connection, I need to cast from and to to type AbstractFilter, as that's the class with the methods needed to build the list. Heh, I suppose I could put the method signatures into ConnectTo and ConnectFrom, couldn't I? Then I could skip the cast and also stop feeling guilty about using marker interfaces!

Actually, I don't think you do need to do that. What about making them protected? If they're nested inside Connector (which I assume is final), that shouldn't be a problem, and outside Connector, they'll be package-private.

I think the warning is saying that you're showing details that clients wouldn't normally see - ie, it's kind of a "documentation" warning - but protected items are normally visible anyway, so it shouldn't be a problem.

Give it a bash. It might just work ... but don't sue me if it doesn't.

[Edit] Alternatively:
1. Add an instanceof AbstractFilter check in the method. It's not ideal, but at least if it succeeds you know you can do the cast without any annotations.
2. Add a '@throws ClassCastException' tag to say that an "illegal" From or To object was passed (probably better).
or maybe both.

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:What about making them protected? If they're nested inside Connector (which I assume is final), that shouldn't be a problem, and outside Connector, they'll be package-private.


I made them all static protected nested interfaces inside Connector. Works great, except that when I make Connector final, I still get the "exporting" warning. Leaving Connector non-final makes the warning go away. I'd have expected the opposite. What's going on?
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I've created a new topic that asks this question, so people already bored by my type-safety issues might look at it anew.
 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:I'll probably regret this (looking at you, Stephan van Hulst ),



Reaching back to your previous topic about the matter, didn't your classes already implement In and Out?

 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
They did, but there were so few methods declared in those (and it ultimately proved itself to be beyond my abilities to create an InOut by composing an In and Out, since the former must be able to call the methods of the latter) that it became much simpler to consolidate them into a single class, and just have subclasses use only the superclass functionality they needed. This left me with the problem this thread is about: enforcing type-safety when there can be a lot of legal combinations of derived classes, but not all combinations are legal. The marker interface method kind of brings me full-circle to where my legal combinations are, in a sense, all Out classes on the left, and all In classes on the right (in actual practice, the legal combinations are: (PushOut, PushIn), (PullOut, PullIn), but the problem is essentially the same). So, in a way, I'm ending up where I started, but with a much cleaner approach, since all my classes are derived from a single superclass, yet I can differentiate them for type-safety purposes with a small number of marker classes, and (with your approval, I hope), no visibility tricks, other than that the top-level abstract superclass is package-private.
 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
To me it seems everything can be resolved if you treat *everything* as a Filter, and a filter isPassive() or isActive() (pull/push), and a filter isSource() or isSink() (or both).

The RandomAccess marker interface was already mentioned. A much better solution would have been if the List and Map interfaces had isRandomlyAccessible().
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:The RandomAccess marker interface was already mentioned. A much better solution would have been if the List and Map interfaces had isRandomlyAccessible().

Hmmm. Not so sure about that. That turns the business of "being randomly accessible" into a procedure (or attribute), not something that's an intrinsic part of the type. Personally, I rather like being able to restrict a method I write to only accept a random-access List - which, of course, is much easier now with generics.

I certainly agree that marker interfaces should be used sparingly, but I don't think they're always wrong; and they do offer a type-based alternative for situations that might otherwise be tough to implement in single-inheritance. For example: if RuntimeException had been a marker interface instead of part of the Exception hierarchy, I think it would have made things a lot more flexible.

My 2¢. YMMV.

Winston
 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:That turns the business of "being randomly accessible" into a procedure, not something that's an intrinsic part of the type.

I see what you mean. If this is the case, I much rather prefer the use of annotations instead.

For instance, IllegalArgumentException could have been declared to be unchecked with an annotation, and then we could do away with RuntimeException (which is poorly named anyway).

We could also have gotten rid of Throwable and Error, and just make everything Exception.

An annotation could have had the added benefit of switching off exception checking for swaths of code, not just particular exception types:
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:To me it seems everything can be resolved if you treat *everything* as a Filter, and a filter isPassive() or isActive() (pull/push), and a filter isSource() or isSink() (or both).


That's ultimately what I did. I held off on the strength of your advice that my superclass shouldn't include functionality not used by every subclass, but maybe I misunderstood what you were telling me. As it stands now, all Sources and Sinks are actually subclasses of AbstractFilter. The Sources just have null as their own source, and the Sinks have null as their sinks. The AbstractFilter is package-private, while the Source, Sink, and Filter classes (all being subclasses of AbstractFilter) are public. The (now public) marker interfaces give me compile-time type-safety regarding which classes can be connected, without directly exposing AbstractFilter to client code. Seems to work, and (I think) I understand it.
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:
Winston Gutkowski wrote:That turns the business of "being randomly accessible" into a procedure, not something that's an intrinsic part of the type.

I see what you mean. If this is the case, I much rather prefer the use of annotations instead.


I've tried a couple of times to read up on annotations, but never seem to quite get what they are, much less how I can use them myself. But, a couple of folks have now suggested this might be a good tool to use in solving my type-safety issue. Can you folks suggest a truly simple tutorial? I can handle most computer programming topics pretty readily, but annotations seem to be a particular problem for me.
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:I see what you mean. If this is the case, I much rather prefer the use of annotations instead...

You may be right there. I certainly like them for things like @Override and @SafeVarargs, but I have to admit to still being very ignorant about making my own custom ones, or how I would use them in a case like Stevens'.

My main concern (and his, I think) was that the solution should be type-based, not code- (or procedure-) based - ie, he wanted to prevent someone from supplying an invalid value, not "handle" it once it had already happened.

However, you've got me thinking. I really must read up a bit more on annotations and trying creating a few myself ...

Winston
 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:I held off on the strength of your advice that my superclass shouldn't include functionality not used by every subclass, but maybe I misunderstood what you were telling me. As it stands now, all Sources and Sinks are actually subclasses of AbstractFilter. The Sources just have null as their own source, and the Sinks have null as their sinks.

No, that still sounds iffy to me :P

The AbstractFilter is package-private, while the Source, Sink, and Filter classes (all being subclasses of AbstractFilter) are public.

This is unfortunate. Source, Sink and Filter sound like they should be interfaces, and AbstractFilter sounds like something that should (partially) implement Filter, Source and Sink.

The (now public) marker interfaces give me compile-time type-safety regarding which classes can be connected, without directly exposing AbstractFilter to client code. Seems to work, and (I think) I understand it.

Yes, you can do this with annotation processing. This is an operation on the source itself. I'm convinced however, that you can do this without marker interfaces, annotations or weird inheritance trees.

I've tried a couple of times to read up on annotations, but never seem to quite get what they are, much less how I can use them myself. But, a couple of folks have now suggested this might be a good tool to use in solving my type-safety issue.

Annotations are literally what it says on the tin: It's as if your code is on a sheet of paper and you pen down extra comments about the code. As such, annotations shouldn't really be used to define behavior at runtime, but are rather used by tools that do something with your code, such as compilers and application containers.

I don't think annotations are applicable for you application, for the same reason marker interfaces aren't applicable: You need to match a source and a sink, because you can do one thing with the one, and another thing with the other. That behavior should be declared within their respective interfaces! I really still don't understand why a Source doesn't just declare methods to get something from it, and Sink has methods to put something in it. You can also make them push-based by adding events to these interfaces.
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:This is unfortunate. Source, Sink and Filter sound like they should be interfaces, and AbstractFilter sounds like something that should (partially) implement Filter, Source and Sink.
...
I really still don't understand why a Source doesn't just declare methods to get something from it, and Sink has methods to put something in it. You can also make them push-based by adding events to these interfaces.

I agree (in principle) with your second point, but disagree with the first, because in Stevens' "world" a Filter is both a Source and a Sink - or rather, can either consume or produce whatever it is these things work with (and maybe Consumer and Producer, or Input and Output are better names).

So perhaps the question for Stevens should be: What is a Connector? Right now, it's just a non-instantiable class with a static method in it - ie, a utility class - but if it was an actual object, then you could connect an Input and Output to it (or supply them to a constructor) or "register" them the way you do a Listener with a component.

I think maybe we need a bit more background on what these things actually are (or do) before we leap to too many conclusions, but (@Stevens) as you can see, there are many possibilities, and maybe 'markers' aren't the answer.

I'd still say they're a reasonable solution to the problem as it was stated earlier though.

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan van Hulst wrote:
Stevens Miller wrote:I held off on the strength of your advice that my superclass shouldn't include functionality not used by every subclass, but maybe I misunderstood what you were telling me. As it stands now, all Sources and Sinks are actually subclasses of AbstractFilter. The Sources just have null as their own source, and the Sinks have null as their sinks.

No, that still sounds iffy to me :P

The AbstractFilter is package-private, while the Source, Sink, and Filter classes (all being subclasses of AbstractFilter) are public.

This is unfortunate. Source, Sink and Filter sound like they should be interfaces, and AbstractFilter sounds like something that should (partially) implement Filter, Source and Sink.


Well, now I'm confused again. If Source, Sink, and Filter are interfaces, and AbstractFilter only partially implements them, what implements the rest?
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:Well, now I'm confused again. If Source, Sink, and Filter are interfaces, and AbstractFilter only partially implements them, what implements the rest?

I think Stephan's simply trying to work out how they fit together. Right now, you have a private implementation (AbstractFilter), which is the parent to your three public classes, one of which (Filter) sounds as if - by convention, at least - it ought to be the interface that AbstractFilter is implementing.

Perhaps if you could explain to us in layman's terms (and sorry if this seems like repetition), what these classes are trying to do, it might give us a better handle on the design.

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:Perhaps if you could explain to us in layman's terms (and sorry if this seems like repetition), what these classes are trying to do, it might give us a better handle on the design.


I really appreciate all the effort you guys are putting into helping me with this. Let me try to explain it without having to ask you to read the code (though, maybe, that would actually be simplest).

I'll start with the "push" model, since I think that's naturally the easier to envision:

A Source generates references to instances of a class (in my case, the instances are actually frames of video, but they could be to anything and the system I'm actually coding uses generics to allow for references to anything the client wants to use). When it is ready to produce the next reference (for example, when a camera has captured the next frame of video, which it does once very 1/30th of a second), it calls a method of its Sink, called pushData(D d), where D is the class we're passing references to. Note that the Source generates these references on a thread it created when the Source's start() method was called. So, once started, everything I'm talking about here runs on that thread.

In a very simple situation, the Sink would do some kind of processing on the data in the instance whose reference it received, by calling its goData(D d) method from within its pushData(D d) method, then pushData(D d) would return. In practice, the Sink will be a client's subclass of the Sink I am writing, so the client can override goData(D d) (or, if my Sink is abstract, it will implement goData(D d), to do something useful). An example would be a Source that sends video frames to a Sink whose goData(D d) method painted those frame onto a JPanel, thus displaying the video in real time. The Source's streaming loop might look something like this:



And a simple Sink's pushData(D d) method might look something like this:



Not too complicated, right? But, not very versatile or interesting, either. The fun begins when the first Source's Sink isn't the end of the line. A Filter, being both a Sink and Source, can do something interesting with the data it receives (for a frame of video, for example, it might swap the red and blue pixel values, for a special effect), then, instead of merely returning, it calls its own Sink's pushData(D d) method, sending the processed data down the line to the next Sink (which might be yet another Filter, and so on).

So, a Filter's pushData(D d) method might look something like this:



So, we can see that both Sources and Filters must have the getMySink() method, which suggests that method belongs in a superclass common to both of them.

But, now let me mix it up a bit and discuss the "pull" model, which is very similar, but works by the Sink making calls for the data, instead of the Source calling with the data.

In the "pull" model, the Sink calls its Source's pullData() method:



The code above, running in a Sink, is calling its Source's pullData() method, which might be as simple as this:



Again, however, a Sink's Source might be a Filter, with a pullData() method like this:



Here, we see that both Sinks and Filters have the getMySource() method, so maybe that also belongs in a common superclass.

But, to be as versatile as we will need it to be, this system must be able to mix "push" model Sources and Sinks with "pull" model Sources and Sinks. How can it do that? With Filters that are "push" model Sources and "pull" model Sinks (and vice versa, but ignore that for now).

A Filter that is a "push" model Source would have a pushData(D d) method. A Filter that is a "pull" model Sink would call its own Source's pullData() method. It would alternate between those two in a loop, like this:



(That could be one line, getMySink().pushData(getMySource().pullData());, but I broke it out for clarity.)

And here's where it all becomes too much for me: the code above shows that a Filter might need both a getMySource() method and a getMySink() method. Thus, if those two methods are going to be in a superclass of Filter, then that superclass must also be the superclass of Source, and the superclass of Sink. Since you can't have two superclasses in Java, I ended up putting those (and a number of other methods that at least two of Source, Sink, and Filter share) into AbstractFilter, with the result being that there is functionality in AbstractFilter that all three of its subclasses inherit, but not all three use. No Source that is only a Source (and not a Filter), ever needs to call getMySource(), and no Sink that is only a Sink (and not a Filter), ever needs to call getMySink(). But, both Filters and Sources have to be able to call getMySink(), and both Filters and Sinks have to be able to call getMySource().

Thus my dilemma: I have three classes and two methods, with both methods being used by two, but not three of the classes, but not the same two classes; one class uses both methods, while each of the other two uses one or the other method. This would be a perfect case for multiple inheritance, but Java doesn't allow that. So, I merged what would have been the two superclasses into one, leaving me with a class that has more functionality in it than all three of its subclasses need.

If there's a better way, I'd trade free legal advice to know what it is.
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
I know you asked for a layman's description, but, expressed as a problem in pure Java, apart from what I'm trying to do, the question is: How might I refactor the following code so the getMySource() and getMySink() methods are defined only once each?

 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:I know you asked for a layman's description, but, expressed as a problem in pure Java, apart from what I'm trying to do, the question is: How might I refactor the following code so the getMySource() and getMySink() methods are defined only once each?...

This is one of the reasons we (or at least I) advise people to turn their computers OFF when they're designing.

I think the problem here is that you already have an idea in your head about what you want things to look like in Java, and you're trying to make the design fit that pattern.
But going back to your description above, I have a few questions:

1. Why does the framework have to support both "push" and "pull" models? Clearly you have something that is spitting out objects, and something that accepts them, but does either "thing" really care what model is used to process them?

2. It strikes me that this framework is missing generics. Might it not be better to have the Source specifiy what it's spitting out? - ie: Source<T> (or in your case, Source<VideoFrame>).

3. Are you on version 8? Because I have a feeling that this problem might fit very nicely into the Stream/lambda or Stream/function model. Unfortunately, I haven't read up too much on it myself yet, so I can't really advise you exactly how; but it certainly "smells" like Stream might be a good fit. And then you don't have to worry about "sources" and "sinks".

Alternatively, what about a Deque? Java already has a few existing implementations, both blocking and non-, and all you'd need to do to create a Filter (which appears to be the only thing that actually "changes" objects) is add some kind of "function" that performs the process either (a) when an object is placed in the queue, or (b) when it's "pulled" from it (possibly better).

Then your model would be something like: SourceDeque (or Filter) → Sink - and the only thing you might want in such a set-up is a VanillaFilter that doesn't do any processing at all.

And just to re-iterate: the above are just suggestions - food for thought - and I'm sure that Stephan may have other ideas.

HIH. Fun thread.

Winston
 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
So really, you want a pipeline of filter operations on a video frame. The pipeline can both push a frame to a sink when it becomes available at the source, or it can pull a frame from a source when when the sink requires it.

So far so good, this is not difficult, and also doesn't require complex code.

I think you're running into problems because you're trying to use inheritance to make different kinds of filters. Instead, use composition to separate the pipelining code from the actual filtering operations:

 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Now, you can easily create different kinds of filters by just calling any of the createXFilter methods with the appropriate filtering operations.

You can string filters together into a pipeline by using filters as the suppliers or consumers in other filters. At the ends, you can use any kind of supplier or consumer as the source and sink of your pipeline, even a lambda:


 
Stephan van Hulst
Saloon Keeper
Posts: 7812
142
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
You can make stringing filters together safe using a helper class:
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:I think the problem here is that you already have an idea in your head about what you want things to look like in Java, and you're trying to make the design fit that pattern.


You're an insightful fellow, Winston. Yes, this is all based on an existing system that has been part of the Windows API for a long time. I'm trying to generalize it, though. The Windows tool is for media, but the Source/Filter/Sink approach could be used for all kinds of processing/modeling. I've written a lot of C++ code that uses it, but someone hereabouts convinced me, a year or two back, that I was underestimating Java's ability to do the same thing (that was you, btw ), so I'm creating the same framework to let me port my C++ to Java.

1. Why does the framework have to support both "push" and "pull" models? Clearly you have something that is spitting out objects, and something that accepts them, but does either "thing" really care what model is used to process them?


Some things work best in one model, others work best in the other model. You could, I suppose, commit yourself to one model and not lose the ability to create any applications. It's just that some stuff fits naturally into one model or another. For example, video cameras are natural "push" Sources, as they will want to send a frame of video somewhere every 1/30th of second. Trying to pull a frame from a camera means you are blocked on the camera, which commits you for 33.333 ms (or longer, if the camera decides to shutdown without telling you). Conversely, post-processing tools (format converters, for example), are natural "pull" Sinks, as they will take however long they need to process a frame, then ask for more as soon as they can. In a lot of cases, their Sources will be able to have the next frame ready before the Sink needs it, so the Sink never blocks on a pull call. Add to this that any of these things can be multi-threaded, and the need to let the client programmer make their own choices about pushing and pulling becomes apparent. (I've also implemented a multi-threaded PushSource/PullSink filter that, combined with the PullSource/PushSink filter described in my previous comments, effectlvely lets you turn any pusher into a puller, and any puller into a pusher and, yes, discussing this stuff at any length does provoke a lot of snickers and vulgar puns; professional hazard, apparently.)

2. It strikes me that this framework is missing generics. Might it not be better to have the Source specifiy what it's spitting out? - ie: Source<T> (or in your case, Source<VideoFrame>).


Absolutely! My existing code does precisely that. I left it out to keep my examples pared down to the barest essentials necessary to describe the specific problem I'm addressing just now.

3. Are you on version 8? Because I have a feeling that this problem might fit very nicely into the Stream/lambda or Stream/function model. Unfortunately, I haven't read up too much on it myself yet, so I can't really advise you exactly how; but it certainly "smells" like Stream might be a good fit. And then you don't have to worry about "sources" and "sinks".


Yes, I'm on 8. Using lambdas and method references, even . Streams might do what I need, and I have Horstmann's "Java SE 8 for the Really Impatient" here on my desk. I'll be reading up on those in the near future. Here, though, it seems to me that the question of how to refactor classes that share some, but not all, of a few methods, is not limited to the context of my Source/Filter/Sink situation, and I'm looking for help on how to tackle that aspect of it, so I'll know what to do if I encounter it again, in another context.

Alternatively, what about a Deque? Java already has a few existing implementations, both blocking and non-, and all you'd need to do to create a Filter (which appears to be the only thing that actually "changes" objects) is add some kind of "function" that performs the process either (a) when an object is placed in the queue, or (b) when it's "pulled" from it (possibly better).


Sure, I'm not too proud to use someone else's code if it will get the job done. At this point, it's really about the refactoring for me, not the project I'm working on. That is, I have learned a lot about OOP in the last couple of years, but this particular business of how best to use Java to solve a problem I'd solve with multiple inherited superclasses in C++ is something I have to admit I still don't know.

HIH. Fun thread.


Glad you think so! I think this is an interesting problem, and feel it's been worth my own time to stop, think, and explore implementation options. But I also suspect that you more experienced folks wouldn't see it as being quite as daunting as it appears to me (at the moment, anyway). I'm delighted it interests folks like you and Stephan enough to be worth advising me on it.
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stephan, thanks for all that detailed code! I'll have to study it a bit before commenting to any significant degree (and I will do that), but it does have some similarities to what I am already doing. You're using factory methods where I am inheriting, but, either way, it seems to me that we both have a main class that can serve as a Filter, Source, or Sink, and that, in the latter two cases, calling the methods that are unique to the other generates an exception (Lines 27 and 37 of your Filter class). I do much the same thing, but I thought that was what you were advising against when you suggested that writing functionality into a superclass that isn't used by all subclasses was a bad idea. Granted, you're not subclassing: you're using your factory methods to pass Filter objects configured as Suppliers (Sources) or Consumers (Sinks) to callers that will assign those references to variable of those types. I do the same thing, just using abstract subclasses that only implement that part of the Filter's methods that a Source or Sink needs. Clients can't call the unused functionality because it's not part of the interface available to them when they subclass an abstract Source (or abstract Sink). At the moment, it looks like two ways of, more or less, doing the same thing.

But, like I said, I want to take some time and study what you've posted, before going on at any further length.

Thanks a lot!
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
btw, I see that you, Stephan, are in Enschede, The Netherlands, and Winston is in London, so you guys are six and five hours ahead of me, respectively. Creates an interesting asychrony to this conversation we are having. Here, it's 1:00 pm as I write this. For you guys, it's 7:00 and 6:00. A fair number of our comments go up when at least one or two of us are asleep (or should be). Yet, this conversation is almost as effective as if we were in the same time and place (maybe more so, since I have to stop and think before I say anything, which I should probably do a lot more often, come to think of it).

Certainly proves that all those naysayers who predicted the internet would isolate us all from each other were dead wrong.
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:but someone hereabouts convinced me, a year or two back, that I was underestimating Java's ability to do the same thing (that was you, btw )...

Ooof. Hoisted by me own petard.

Have you had a look at Stephen's solution? It seems pretty good to me - though perhaps not with quite the same set of classes you may want. He also uses the terms Producer, Consumer and Pipeline, which are probably the most common formal descriptions of this general problem, so you might find some alternatives by Googling using those terms. Indeed, a Google of "producer consumer pattern" got me this page - which is probably worth reading - as the very first link.

However, I definitely suggest you also look at Streams. For one thing, because they're declarative, it leaves a pile of scope for things like "parallelizing" (ugh), which may be extremely useful for something like video.
Wouldn't you prefer to have all four of your CPUs working on a "pixellating" job rather than just one? And with Streams it's automatic - no ghastly Threads or "marshalling" to worry about.

Sorry I can't be more definite, but I fear I've hit the limits of my useful knowledge.

Best of luck with it - and I'd be interested to see what you come up with.

Winston
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:btw, I see that you, Stephan, are in Enschede, The Netherlands, and Winston is in London...

My SP maybe (British Telecom), but I'm 400 miles away in the wilds of Scotland, where it's-a-blowin' something fierce tonight.

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
To go on a a bit more about why I'm not using Deque or something similar: one reason is I'm not familiar with those options enough to know if they can help me when I get to some aspects of this that require more than a linked list or queue. When doing video work, one often needs to take one frame in, and produce two or more frames out, using something akin to the Unix "tee" utility. Likewise, one sometimes needs to take two or more frames in, and combine them into a single frame to go out (in a video studio, this is done with a "mixer," same as in an audio studio). For this reason, the data structures Windows maintains for all of this are not called queues or lists: they are called graphs.

A diagram of how this all might look could follow these conventions:

1. A node in the graph looks like this: [ ]
2. Data always flows from left to right.
3. A node that is a source has an edge connecting it from its right side to its sink's left side.
4. A node that is a sink has an edge connecting it from its left side to its sink's right side.
5. An edge that connects a push source to a push sink is a directed edge pointing at the sink.
6. An edge that connects a pull source to a pull sink is a directed edge pointing at the source (but remember that data always flows from left to right).

Thus, the simplest setup would look like this:

That's a push source sending data to a push sink.

A pull sink getting data from a pull source is almost as simple:

One simply has to remember that the arrow indicates who calls whom to move data, not the direction in which data flows.

A push source feeding a filter that is a push source to a sink is this:

With a little multi-threading, one can create a filter that is a push sink, and a pull source:

If you want to feed multiple sinks, you need a tee, or other filter that turns one frame into two or more:

In a situation where you have, say, two cameras, a filter that sets the alpha (transparency) of a pixel based on a key color, another that mixes the two video signals based on that alpha, and a main display as well as a raw video display from each camera, it starts getting more interesting (and illustrates why the structure is a graph, not a list):

I actually have the above working, but it's in a gawdawful gas-factory of badly structured code, so my current efforts are all directed towards implementing it in something more maintainable. There may be existing systems out there that can handle this, but I often find I move faster if I roll my own, than if I try to learn yet another API. If not, at least I tend to learn more that is of general applicability that way.
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:My SP maybe (British Telecom), but I'm 400 miles away in the wilds of Scotland, where it's-a-blowin' something fierce tonight.

Scotland? Whereabouts? I was fortunate enough to take a business trip to Aberdeen, about twelve years ago. Over the weekend, I met a lot of friendly people who spoke a language I could almost understand . It was summer, and I could see light in the sky all the way to 10:30 pm. Very memorable, enjoyable trip, that was.
 
Winston Gutkowski
Bartender
Posts: 10573
65
Eclipse IDE Hibernate Ubuntu
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Stevens Miller wrote:Scotland? Whereabouts? I was fortunate enough to take a business trip to Aberdeen...

Other side (West coast). A little place called Girvan, about a third of the way from Ayr to Stranraer. Lovely beach and scenery - including Ailsa Craig, where the granite for just about every competition curling stone ever made up to about 20 years ago came from (it's now a bird sanctuary) - but wind is pretty much a constant.

Winston
 
Stevens Miller
Bartender
Posts: 1444
30
C++ Java Netbeans IDE Windows
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Winston Gutkowski wrote:A little place called Girvan...

Just looked it up on Google Maps. That is rather bucolic. How does a software guy make a living in a place like that? And, are they hiring?
 
Campbell Ritchie
Marshal
Posts: 55741
163
  • Mark post as helpful
  • send pies
  • Quote
  • Report post to moderator
Girvan the wilds? Troon may be in the wilds, but surely not Girvan.

Maybe we should get back to the official subject of the thread.
 
It is sorta covered in the JavaRanch Style Guide.
  • Post Reply Bookmark Topic Watch Topic
  • New Topic
Boost this thread!