Peter Wippermann

Interesting Task! I hope Claire Bird didn't give up?!
I was eager to solve this problem by myself, so here's my solution. I found AggregatingIterator a better name for that class.

Feel free to comment ;-)

package org.javaranch; import java.util.ArrayList; import java.util.Collection; import java.util.Collections; import java.util.Comparator; import java.util.Iterator; import java.util.List; import java.util.NoSuchElementException; /** * <p> * Implementation of an {@link Iterator} that returns a SORTED stream of results from a collection of input {@link Iterator}s, where the values in the input * iterators are also sorted. * </p> * * <p> * The values returned by the input iterators WILL ALWAYS BE SORTED according to the same {@link Comparator} that is specified in {@link AggregatingIterator}'s * constructor. The input iterators will never contain <tt>null</tt> * </p> * * <p> * The input iterators are expected to have a very large (or possibly infinite) set of results, so this implementation returns results AS SOON AS THEY ARE * AVAILABLE, rather than waiting until the input iterators are drained before returning results. * </p> * * <p> * This class can also eliminate duplicate values from the input iterators (making all output values distinct), if the <tt>eliminateDuplicates</tt> flag is set * in the constructor. Note that the specified {@link Comparator} is used to determine if two objects are identical, rather than the objects' * {@link #equals(Object)} methods. * </p> * * <p> * Note: this class is <em>not</em> thread-safe, and cannot be called concurrently from multiple threads * </p> * */ public class AggregatingIterator<T> implements Iterator<T> { /** * Extends any backing iterator by a buffer that stores the last requested value from next().<br> * Allows to ask the iterator multiple times for its value without requesting the next value. * * @param <E> */ private static class BufferingIterator<E> implements Iterator<E> { private Iterator<E> backingIterator; private E bufferedValue = null; public BufferingIterator(Iterator<E> backingIterator) { super(); this.backingIterator = backingIterator; } public E getBufferedValue() { return bufferedValue; } @Override public boolean hasNext() { return backingIterator.hasNext(); } /** * Retrieves the next value from the backing iterator and buffers it. * * @return the new buffered value; */ @Override public E next() { bufferedValue = backingIterator.next(); return bufferedValue; } @Override public void remove() { backingIterator.remove(); } } /** * Gives us the possibility to sort the iterators by their current buffered value. */ private final Comparator<BufferingIterator<T>> iteratorComparator = new Comparator<BufferingIterator<T>>() { @Override public int compare(BufferingIterator<T> o1, BufferingIterator<T> o2) { return comparator.compare(o1.getBufferedValue(), o2.getBufferedValue()); } }; private final boolean eliminateDuplicates; private final Comparator<T> comparator; private final List<BufferingIterator<T>> iterators; private T lastValueReturned = null; private T nextValueToReturn = null; private boolean hasBeenInitialised = false; /** * <p> * Creates a {@link AggregatingIterator} with the specified collection of input {@link Iterator}s and {@link Comparator}, eliminating duplicates if required. * * <p> * This constructor must NOT call the {@link Iterator#next()} or {@link #hasNext()} methods of the input iterators, as these methods may block, and the * constructor needs to return as soon as possible. * * @param iterators * The collection of input iterators, over which the results will be returned. The values in these iterators WILL ALWAYS BE SORTED by the * specified {@link Comparator}. * * @param comparator * The {@link Comparator} that is to be used to compare items. The output values returned from {@link #next()} will be ordered by this * comparator, and the values in the input iterators should also be ordered by it. * * @param eliminateDuplicates * If this is set to <tt>true</tt> then the output values will contain no duplicate values, according to the specified {@link Comparator}. */ public AggregatingIterator(final Collection<? extends Iterator<T>> iterators, final Comparator<T> comparator, final boolean eliminateDuplicates) { this.eliminateDuplicates = eliminateDuplicates; this.comparator = comparator; this.iterators = new ArrayList<BufferingIterator<T>>(iterators.size()); for (Iterator<T> currentIterator : iterators) { this.iterators.add(new BufferingIterator<T>(currentIterator)); } } /** * @see Iterator#hasNext() */ @Override public boolean hasNext() { if (nextValueToReturn != null) { return true; } if (!hasBeenInitialised) { initialise(); } findNextValue(); return nextValueToReturn != null; } /** * Returns the next value from the input iterators, where values are returned <em>in order</em>, according to the {@link Comparator} supplied in the * {@link #AnIterator(Collection, Comparator, boolean)} constructor. * * @throws NoSuchElementException * if there are no more values to return */ @Override public T next() { if (!hasNext()) { throw new NoSuchElementException("No more values!"); } // hasNext() will have initialised the next value lastValueReturned = nextValueToReturn; nextValueToReturn = null; return lastValueReturned; } private void findNextValue() { nextValueToReturn = null; if (iterators.isEmpty()) { return; } Collections.sort(iterators, iteratorComparator); BufferingIterator<T> iteratorWithSmallestValue = iterators.get(0); T smallestValueFound = iteratorWithSmallestValue.getBufferedValue(); if (lastValueReturned == null || !eliminateDuplicates) { nextValueToReturn = smallestValueFound; iterateOrRetire(iteratorWithSmallestValue); } else { // This is not the first value to return, so we have to check for duplicates if (!isAGreaterThanB(smallestValueFound, lastValueReturned)) { // Skip this value and retry iterateOrRetire(iteratorWithSmallestValue); findNextValue(); } else { nextValueToReturn = smallestValueFound; } } } /** * Iterates the iterator to the next value - or retires it, if it has no more values. * * @param iteratorWithSmallestValue */ private void iterateOrRetire(BufferingIterator<T> iteratorWithSmallestValue) { if (iteratorWithSmallestValue.hasNext()) { iteratorWithSmallestValue.next(); } else { iterators.remove(iteratorWithSmallestValue); } } /** * Initialises all BufferingIterators.<br> * Honours the requirement, that AnIterator's constructor may not call next() or hasNext(). So initialisation is done late. */ private void initialise() { for (BufferingIterator<T> currentIterator : iterators) { if (currentIterator.getBufferedValue() == null) { iterateOrRetire(currentIterator); } } hasBeenInitialised = true; } /** * @param A * @param B * @return true, if A is greater than B */ private boolean isAGreaterThanB(T a, T b) { return comparator.compare(a, b) > 0; } /** * Calling this method is not supported, and throws a {@link UnsupportedOperationException} */ @Override public void remove() { throw new UnsupportedOperationException("Calling remove() is not supported"); } }

Here's a unit test inspired by one of the answers before.

package org.javaranch; import java.util.ArrayList; import java.util.Collections; import java.util.Comparator; import java.util.Iterator; import java.util.List; import org.junit.After; import org.junit.AfterClass; import org.junit.Assert; import org.junit.Before; import org.junit.BeforeClass; import org.junit.Test; public class AggregatingIteratorTest { @BeforeClass public static void setUpBeforeClass() throws Exception { } @AfterClass public static void tearDownAfterClass() throws Exception { } @Before public void setUp() throws Exception { } @After public void tearDown() throws Exception { } public static final Comparator<Integer> comparator = new Comparator<Integer>() { @Override public int compare(Integer obj1, Integer obj2) { // TODO Efficiency of this implementation is questionable Integer thisInteger1 = new Integer(obj1); Integer thisInteger2 = new Integer(obj2); return thisInteger1.compareTo(thisInteger2); } }; @Test public void testEliminateDuplicates() { ArrayList<Iterator<Integer>> iterators = new ArrayList<Iterator<Integer>>(); List<Integer> numbers = new ArrayList<>(); // First iterator numbers.add(1); numbers.add(4); numbers.add(5); numbers.add(5); // Requirements say: "The values returned by the input iterators WILL ALWAYS BE SORTED" Collections.sort(numbers, comparator); Iterator<Integer> it = numbers.iterator(); iterators.add(it); // Second iterator numbers = new ArrayList<>(); numbers.add(6); numbers.add(4); numbers.add(1); Collections.sort(numbers, comparator); it = numbers.iterator(); iterators.add(it); checkResults(new int[] { 1, 4, 5, 6 }, new AggregatingIterator<>(iterators, comparator, true)); } @Test public void testKeepDuplicates() { ArrayList<Iterator<Integer>> iterators = new ArrayList<Iterator<Integer>>(); List<Integer> numbers = new ArrayList<>(); // First iterator numbers.add(1); numbers.add(4); numbers.add(5); numbers.add(5); // Requirements say: "The values returned by the input iterators WILL ALWAYS BE SORTED" Collections.sort(numbers, comparator); Iterator<Integer> it = numbers.iterator(); iterators.add(it); // Second iterator numbers = new ArrayList<>(); numbers.add(6); numbers.add(4); numbers.add(1); Collections.sort(numbers, comparator); it = numbers.iterator(); iterators.add(it); checkResults(new int[] { 1, 1, 4, 4, 5, 5, 6 }, new AggregatingIterator<>(iterators, comparator, false)); } private void checkResults(int[] expected, AggregatingIterator<Integer> iterator) { int i; for (i = 0; iterator.hasNext(); i++) { int curVal = iterator.next().intValue(); Assert.assertEquals(expected[i], curVal); System.out.println(curVal); } Assert.assertEquals("Iterator had returned less elements than expected!", expected.length, i); } }

Ted Dunning wrote:Not really. We had an implementation in Pig contributed a long time ago, but nobody really had time to bring it up to snuff.

We do have a very reasonable parallel implementation of LDA, however, which is often considered the next generation algorithm after PLSI.

Thanks. LDA sounds interesting, I'll have a look into it.

I was looking for PLSI in Mahout a few years ago. Besides that there was no out-of-the-box implementation for PLSI, I didn't found anything in Mahout that would help me to implement it by myself.
So has anything changed ever since? An implementation or at least some support in Mahout?

Thanks :-)

Hi folks,
hi Federico,

how can businesses choose GWT as their strategic toolkit to render web pages?
In my opinion such investments are dangerous, because I would not say that Google drives a continous business. indeed, they are very innovative and not afraid to try something new.

But they also don't hesitate to take projects off, if they don't succeed or at least not as much Google would like them to do.
Google Wave was a nice example for that.

So can I recommend GWT to clients, although its continuation may be in question - or is GWT already mature enough for business, even if it wouldn't be brought further by google?

TIA for your answers,
Peter

Oh I see, good addition. I hope that helps others, reading this later on.

Ok I got it, let me have a look on that tomorrow. Stay tuned ;-)

Ok, so your on RAD 7.5 and have an 5.0 EAR, which is targeted for WAS 7, right?
Your in the right perspective and your in the right view? And there is no such submenu JavaEE -> Open WAS Deployment ?

Found the partial answer by my self:
In the JavaEE-Perspektive, go to the Enterprise Explorer view. Right-click on the EAR-Folder and select JavaEE -> Open WAS Deployment
There you can at least find the trusty old "Deployment tab", which I actually needed.

I used to create 1.4 EARs. There in the deployment descriptor editor I had a bunch of tabs including "Module","Security" and "Deployment".
Now when I create a new EAR in version 5.0, the editor only has two tabs "design" and "source". Is there an option to enable the advanced tabs again?

Vyas Sanzgiri wrote:How did you conclude that JSF 1.2 was the problem?

I can't tell yet. My bachelor thesis will be about the evaluation if we could solve our performance issues (better: improve the hardware requirements) by replacing JSF 1.2 with 2.0.

I'm glad to hear from Jason, that my task is likely to succeed and show a real benefit. I will start in March, so unfortunately I don't know enough about the topic and the specific problem we have yet.
I just heared of the promotion here and wanted to take the chance to get some hints for my upcoming thesis, where I could start my analysis. And of course I hope to win a copy of the reference, that would be nice indeed

So is there any JSF implementation I should prefer?
How would you design an examplary application for performance tests?

We're facing some performance issue with our portal and would like to evaluate that.
Do you have any experiences regarding the performance? Or can you even name, what's been done to increase the performance in 2.0?