ArrayList - Reading and writing

[Jimmie Tyrrell]

Got a Java-y question:

I have a loop that executes periodically. The loop iterates over an
Array List, and processes each item.

Randomly, a UDP socket will receive some data, which is then appended
to the Array List.

The problem, naturally, is that my Iterator will throw an exception if
data is received while I'm Iterating over the ArrayList.

Catching the exception is okay - but it occurs while Iterating. I'd
prefer the Iteration process to go untouched, and instead maybe
receive the Exception while _writing_ the data. I could then cache
it, or simply ignore it, whatever.

Any suggestions on how to achieve this? Need to see some code to
understand the problem?

 

[shakah]

Got a Java-y question:

I have a loop that executes periodically.  The loop iterates over an
Array List, and processes each item.

Randomly, a UDP socket will receive some data, which is then appended
to the Array List.

The problem, naturally, is that my Iterator will throw an exception if
data is received while I'm Iterating over the ArrayList.

Catching the exception is okay - but it occurs while Iterating.  I'd
prefer the Iteration process to go untouched, and instead maybe
receive the Exception while _writing_ the data.  I could then cache
it, or simply ignore it, whatever.

Any suggestions on how to achieve this?  Need to see some code to
understand the problem?

Generally you'd synchronize read and write access to the array list,
e.g.:

// ...reader code
synchronized(your_array_list) {
for(java.util.Iterator i=your_array_list.iterator(); i.hasNext(); )
{
your_object tmp = (your_object) i.next() ;
}
}

// ...writer code (UDP)
while(we_wait_for_udp_packets) {
// ...got a packet
synchronized(your_array_list) {
your_array_list.add(new your_object(packet)) ;
}
}

 

[Jimmie Tyrrell]

Generally you'd synchronize read and write access to the array list,
e.g.:

// ...reader code
synchronized(your_array_list) {
  for(java.util.Iterator i=your_array_list.iterator(); i.hasNext(); )
{
    your_object tmp = (your_object) i.next() ;
  }

}

// ...writer code (UDP)
while(we_wait_for_udp_packets) {
  // ...got a packet
  synchronized(your_array_list) {
    your_array_list.add(new your_object(packet)) ;
  }

}

That was perfect. I had tried using synchronized before - wasn't
aware that _both_ operations needed to be synchronized. It seems
overtly logical now

Thanks for the help!

 

[Daniele Futtorovic]

That was perfect. I had tried using synchronized before - wasn't
aware that _both_ operations needed to be synchronized. It seems
overtly logical now

Thanks for the help!

The problem with shakah's approach is that it monopolises the List
Object during the whole process of Iteration -- which may go down well,
but which may also pose some problems.

It would be better to iterate over a copy. You'd still synchronise the
process of copying, as well as the process of adding.

For instance (non-generic code for the sake of simplicity):

List myQueue = ...;

/* adding */
synchronized( someLock ){
myQueue.add(newObject);
}
/* end adding */

/* iterating */
List copy;

synchronized( someLock ){
copy = java.util.Collections.unmodifiableList(myQueue);
}

for(Iterator it = copy.iterator(); it.hasNext(){
//do something involving it.next();
}
/* end iterating */


Note that copying the List doesn't involve copying its /content/.

 

[Daniele Futtorovic]

synchronized( someLock ){
copy = java.util.Collections.unmodifiableList(myQueue);
}

Sorry, that was nonsense (different train of thought). You had rather use
copy = new LinkedList(myQueue);

 

[Daniel Pitts]

Got a Java-y question:

I have a loop that executes periodically. The loop iterates over an
Array List, and processes each item.

Randomly, a UDP socket will receive some data, which is then appended
to the Array List.

The problem, naturally, is that my Iterator will throw an exception if
data is received while I'm Iterating over the ArrayList.

Catching the exception is okay - but it occurs while Iterating. I'd
prefer the Iteration process to go untouched, and instead maybe
receive the Exception while _writing_ the data. I could then cache
it, or simply ignore it, whatever.

Any suggestions on how to achieve this? Need to see some code to
understand the problem?

As some others have pointed out, you need to synchronize the access to
the list.

Now, if it turns out that your UDP handler blocks too long because of
the iteration, you might consider having all the UDP code put the
results into a BlockingQueue (LinkedBlockingQueue or ArrayBlockingQueue,
depending).

Then you can replace all access to the ArrayList with a getter, and the
getter could first drain the blocking queue into the array list before
returning it. That way, you don't need explicit synchronization between
the UDP handling code and the iterating code, the BlockingQueue does it
for you.

 

[Daniel Pitts]

The problem with shakah's approach is that it monopolises the List
Object during the whole process of Iteration -- which may go down well,
but which may also pose some problems.

It would be better to iterate over a copy. You'd still synchronise the
process of copying, as well as the process of adding.

For instance (non-generic code for the sake of simplicity):

List myQueue = ...;

/* adding */
synchronized( someLock ){
myQueue.add(newObject);
}
/* end adding */

/* iterating */
List copy;

synchronized( someLock ){
copy = java.util.Collections.unmodifiableList(myQueue);
}

for(Iterator it = copy.iterator(); it.hasNext(){
//do something involving it.next();
}
/* end iterating */


Note that copying the List doesn't involve copying its /content/.

And what you did doesn't copy the list either, it just wraps it. To copy
it you'd actually have to use new ArrayList(someQueue); The thing is,
that this process ALSO iterates over the whole list (although only for a
fast reference copy).

I had just made a suggestion that is another way to solve the OPs
problem, using new Java 1.5 Concurrency API features.

The approach to take depends on the size of the list, and the required
performance characteristics of the UDP handler.

 

[Daniele Futtorovic]

And what you did doesn't copy the list either, it just wraps it. To copy
it you'd actually have to use new ArrayList(someQueue);

Yes, I'd corrected that in a sucessive post.

The thing is,
that this process ALSO iterates over the whole list (although only for a
fast reference copy).

I had just made a suggestion that is another way to solve the OPs
problem, using new Java 1.5 Concurrency API features.

The approach to take depends on the size of the list, and the required
performance characteristics of the UDP handler.

Now that you mention it -- yeah, his scenario really calls for a
blocking queue. Although I had even named a variable in the sample code
I posted thus, I had focussed my attention too strongly on the iteration
part.

Thanks for improving the answers.


--
I would like to thank all the fish who have taken part in this post.
I hope that other fish will follow the example of those who have
participated, so that, in future, fish all over the world will live
together in harmony and understanding, and put aside their petty
differences, cease pursuing and eating each other and live for a
brighter, better future for all fish, and those who love them.

 

[Daniel Pitts]

Yes, I'd corrected that in a sucessive post.

Yes, I noticed that after the fact. I had 300+ posts to get through, and
thought to reply in the order I read them.

Now that you mention it -- yeah, his scenario really calls for a
blocking queue. Although I had even named a variable in the sample code
I posted thus, I had focussed my attention too strongly on the iteration
part.

It happens to have been a problem that I've had to solve several times,
and each time I had to solve it different ways because of different
constraints.