ArrayList - Reading and writing

Discussion in 'Java' started by Jimmie Tyrrell, Jul 15, 2008.

  1. Got a Java-y question:

    I have a loop that executes periodically. The loop iterates over an
    Array List, and processes each item.

    Randomly, a UDP socket will receive some data, which is then appended
    to the Array List.

    The problem, naturally, is that my Iterator will throw an exception if
    data is received while I'm Iterating over the ArrayList.

    Catching the exception is okay - but it occurs while Iterating. I'd
    prefer the Iteration process to go untouched, and instead maybe
    receive the Exception while _writing_ the data. I could then cache
    it, or simply ignore it, whatever.

    Any suggestions on how to achieve this? Need to see some code to
    understand the problem?
    Jimmie Tyrrell, Jul 15, 2008
    #1
    1. Advertising

  2. Jimmie Tyrrell

    shakah Guest

    On Jul 15, 9:38 am, Jimmie Tyrrell <> wrote:
    > Got a Java-y question:
    >
    > I have a loop that executes periodically.  The loop iterates over an
    > Array List, and processes each item.
    >
    > Randomly, a UDP socket will receive some data, which is then appended
    > to the Array List.
    >
    > The problem, naturally, is that my Iterator will throw an exception if
    > data is received while I'm Iterating over the ArrayList.
    >
    > Catching the exception is okay - but it occurs while Iterating.  I'd
    > prefer the Iteration process to go untouched, and instead maybe
    > receive the Exception while _writing_ the data.  I could then cache
    > it, or simply ignore it, whatever.
    >
    > Any suggestions on how to achieve this?  Need to see some code to
    > understand the problem?


    Generally you'd synchronize read and write access to the array list,
    e.g.:

    // ...reader code
    synchronized(your_array_list) {
    for(java.util.Iterator i=your_array_list.iterator(); i.hasNext(); )
    {
    your_object tmp = (your_object) i.next() ;
    }
    }

    // ...writer code (UDP)
    while(we_wait_for_udp_packets) {
    // ...got a packet
    synchronized(your_array_list) {
    your_array_list.add(new your_object(packet)) ;
    }
    }
    shakah, Jul 15, 2008
    #2
    1. Advertising

  3. On Jul 15, 9:48 am, shakah <> wrote:
    > On Jul 15, 9:38 am, Jimmie Tyrrell <> wrote:
    >
    >
    >
    > > Got a Java-y question:

    >
    > > I have a loop that executes periodically.  The loop iterates over an
    > > Array List, and processes each item.

    >
    > > Randomly, a UDP socket will receive some data, which is then appended
    > > to the Array List.

    >
    > > The problem, naturally, is that my Iterator will throw an exception if
    > > data is received while I'm Iterating over the ArrayList.

    >
    > > Catching the exception is okay - but it occurs while Iterating.  I'd
    > > prefer the Iteration process to go untouched, and instead maybe
    > > receive the Exception while _writing_ the data.  I could then cache
    > > it, or simply ignore it, whatever.

    >
    > > Any suggestions on how to achieve this?  Need to see some code to
    > > understand the problem?

    >
    > Generally you'd synchronize read and write access to the array list,
    > e.g.:
    >
    > // ...reader code
    > synchronized(your_array_list) {
    >   for(java.util.Iterator i=your_array_list.iterator(); i.hasNext(); )
    > {
    >     your_object tmp = (your_object) i.next() ;
    >   }
    >
    > }
    >
    > // ...writer code (UDP)
    > while(we_wait_for_udp_packets) {
    >   // ...got a packet
    >   synchronized(your_array_list) {
    >     your_array_list.add(new your_object(packet)) ;
    >   }
    >
    > }
    >
    >


    That was perfect. I had tried using synchronized before - wasn't
    aware that _both_ operations needed to be synchronized. It seems
    overtly logical now :)

    Thanks for the help!
    Jimmie Tyrrell, Jul 15, 2008
    #3
  4. On 2008-07-15 16:07 +0100, Jimmie Tyrrell allegedly wrote:
    > On Jul 15, 9:48 am, shakah <> wrote:
    >> On Jul 15, 9:38 am, Jimmie Tyrrell <> wrote:
    >>
    >>
    >>
    >>> Got a Java-y question:
    >>> I have a loop that executes periodically. The loop iterates over an
    >>> Array List, and processes each item.
    >>> Randomly, a UDP socket will receive some data, which is then appended
    >>> to the Array List.
    >>> The problem, naturally, is that my Iterator will throw an exception if
    >>> data is received while I'm Iterating over the ArrayList.
    >>> Catching the exception is okay - but it occurs while Iterating. I'd
    >>> prefer the Iteration process to go untouched, and instead maybe
    >>> receive the Exception while _writing_ the data. I could then cache
    >>> it, or simply ignore it, whatever.
    >>> Any suggestions on how to achieve this? Need to see some code to
    >>> understand the problem?

    >> Generally you'd synchronize read and write access to the array list,
    >> e.g.:
    >>
    >> // ...reader code
    >> synchronized(your_array_list) {
    >> for(java.util.Iterator i=your_array_list.iterator(); i.hasNext(); )
    >> {
    >> your_object tmp = (your_object) i.next() ;
    >> }
    >>
    >> }
    >>
    >> // ...writer code (UDP)
    >> while(we_wait_for_udp_packets) {
    >> // ...got a packet
    >> synchronized(your_array_list) {
    >> your_array_list.add(new your_object(packet)) ;
    >> }
    >>
    >> }
    >>
    >>

    >
    > That was perfect. I had tried using synchronized before - wasn't
    > aware that _both_ operations needed to be synchronized. It seems
    > overtly logical now :)
    >
    > Thanks for the help!


    The problem with shakah's approach is that it monopolises the List
    Object during the whole process of Iteration -- which may go down well,
    but which may also pose some problems.

    It would be better to iterate over a copy. You'd still synchronise the
    process of copying, as well as the process of adding.

    For instance (non-generic code for the sake of simplicity):

    List myQueue = ...;

    /* adding */
    synchronized( someLock ){
    myQueue.add(newObject);
    }
    /* end adding */

    /* iterating */
    List copy;

    synchronized( someLock ){
    copy = java.util.Collections.unmodifiableList(myQueue);
    }

    for(Iterator it = copy.iterator(); it.hasNext();){
    //do something involving it.next();
    }
    /* end iterating */


    Note that copying the List doesn't involve copying its /content/.

    --
    DF.
    to reply privately, change the top-level domain
    in the FROM address from "invalid" to "net"
    Daniele Futtorovic, Jul 16, 2008
    #4
  5. On 2008-07-16 01:25 +0100, Daniele Futtorovic allegedly wrote:
    > synchronized( someLock ){
    > copy = java.util.Collections.unmodifiableList(myQueue);
    > }


    Sorry, that was nonsense (different train of thought). You had rather use
    copy = new LinkedList(myQueue);

    --
    DF.
    to reply privately, change the top-level domain
    in the FROM address from "invalid" to "net"
    Daniele Futtorovic, Jul 16, 2008
    #5
  6. Jimmie Tyrrell

    Daniel Pitts Guest

    Jimmie Tyrrell wrote:
    > Got a Java-y question:
    >
    > I have a loop that executes periodically. The loop iterates over an
    > Array List, and processes each item.
    >
    > Randomly, a UDP socket will receive some data, which is then appended
    > to the Array List.
    >
    > The problem, naturally, is that my Iterator will throw an exception if
    > data is received while I'm Iterating over the ArrayList.
    >
    > Catching the exception is okay - but it occurs while Iterating. I'd
    > prefer the Iteration process to go untouched, and instead maybe
    > receive the Exception while _writing_ the data. I could then cache
    > it, or simply ignore it, whatever.
    >
    > Any suggestions on how to achieve this? Need to see some code to
    > understand the problem?

    As some others have pointed out, you need to synchronize the access to
    the list.

    Now, if it turns out that your UDP handler blocks too long because of
    the iteration, you might consider having all the UDP code put the
    results into a BlockingQueue (LinkedBlockingQueue or ArrayBlockingQueue,
    depending).

    Then you can replace all access to the ArrayList with a getter, and the
    getter could first drain the blocking queue into the array list before
    returning it. That way, you don't need explicit synchronization between
    the UDP handling code and the iterating code, the BlockingQueue does it
    for you.

    --
    Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
    Daniel Pitts, Jul 19, 2008
    #6
  7. Jimmie Tyrrell

    Daniel Pitts Guest

    Daniele Futtorovic wrote:
    > On 2008-07-15 16:07 +0100, Jimmie Tyrrell allegedly wrote:
    >> On Jul 15, 9:48 am, shakah <> wrote:
    >>> On Jul 15, 9:38 am, Jimmie Tyrrell <> wrote:
    >>>
    >>>
    >>>
    >>>> Got a Java-y question:
    >>>> I have a loop that executes periodically. The loop iterates over an
    >>>> Array List, and processes each item.
    >>>> Randomly, a UDP socket will receive some data, which is then appended
    >>>> to the Array List.
    >>>> The problem, naturally, is that my Iterator will throw an exception if
    >>>> data is received while I'm Iterating over the ArrayList.
    >>>> Catching the exception is okay - but it occurs while Iterating. I'd
    >>>> prefer the Iteration process to go untouched, and instead maybe
    >>>> receive the Exception while _writing_ the data. I could then cache
    >>>> it, or simply ignore it, whatever.
    >>>> Any suggestions on how to achieve this? Need to see some code to
    >>>> understand the problem?
    >>> Generally you'd synchronize read and write access to the array list,
    >>> e.g.:
    >>>
    >>> // ...reader code
    >>> synchronized(your_array_list) {
    >>> for(java.util.Iterator i=your_array_list.iterator(); i.hasNext(); )
    >>> {
    >>> your_object tmp = (your_object) i.next() ;
    >>> }
    >>>
    >>> }
    >>>
    >>> // ...writer code (UDP)
    >>> while(we_wait_for_udp_packets) {
    >>> // ...got a packet
    >>> synchronized(your_array_list) {
    >>> your_array_list.add(new your_object(packet)) ;
    >>> }
    >>>
    >>> }
    >>>
    >>>

    >>
    >> That was perfect. I had tried using synchronized before - wasn't
    >> aware that _both_ operations needed to be synchronized. It seems
    >> overtly logical now :)
    >>
    >> Thanks for the help!

    >
    > The problem with shakah's approach is that it monopolises the List
    > Object during the whole process of Iteration -- which may go down well,
    > but which may also pose some problems.
    >
    > It would be better to iterate over a copy. You'd still synchronise the
    > process of copying, as well as the process of adding.
    >
    > For instance (non-generic code for the sake of simplicity):
    >
    > List myQueue = ...;
    >
    > /* adding */
    > synchronized( someLock ){
    > myQueue.add(newObject);
    > }
    > /* end adding */
    >
    > /* iterating */
    > List copy;
    >
    > synchronized( someLock ){
    > copy = java.util.Collections.unmodifiableList(myQueue);
    > }
    >
    > for(Iterator it = copy.iterator(); it.hasNext();){
    > //do something involving it.next();
    > }
    > /* end iterating */
    >
    >
    > Note that copying the List doesn't involve copying its /content/.
    >

    And what you did doesn't copy the list either, it just wraps it. To copy
    it you'd actually have to use new ArrayList(someQueue); The thing is,
    that this process ALSO iterates over the whole list (although only for a
    fast reference copy).

    I had just made a suggestion that is another way to solve the OPs
    problem, using new Java 1.5 Concurrency API features.

    The approach to take depends on the size of the list, and the required
    performance characteristics of the UDP handler.
    --
    Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
    Daniel Pitts, Jul 19, 2008
    #7
  8. On 2008-07-20 00:48 +0100, Daniel Pitts allegedly wrote:
    > Daniele Futtorovic wrote:
    >> On 2008-07-15 16:07 +0100, Jimmie Tyrrell allegedly wrote:
    >>>
    >>> That was perfect. I had tried using synchronized before - wasn't
    >>> aware that _both_ operations needed to be synchronized. It seems
    >>> overtly logical now :)
    >>>
    >>> Thanks for the help!

    >>
    >> The problem with shakah's approach is that it monopolises the List
    >> Object during the whole process of Iteration -- which may go down well,
    >> but which may also pose some problems.
    >>
    >> It would be better to iterate over a copy. You'd still synchronise the
    >> process of copying, as well as the process of adding.
    >>
    >> For instance (non-generic code for the sake of simplicity):
    >>
    >> List myQueue = ...;
    >>
    >> /* adding */
    >> synchronized( someLock ){
    >> myQueue.add(newObject);
    >> }
    >> /* end adding */
    >>
    >> /* iterating */
    >> List copy;
    >>
    >> synchronized( someLock ){
    >> copy = java.util.Collections.unmodifiableList(myQueue);
    >> }
    >>
    >> for(Iterator it = copy.iterator(); it.hasNext();){
    >> //do something involving it.next();
    >> }
    >> /* end iterating */
    >>
    >>
    >> Note that copying the List doesn't involve copying its /content/.
    >>

    > And what you did doesn't copy the list either, it just wraps it. To copy
    > it you'd actually have to use new ArrayList(someQueue);


    Yes, I'd corrected that in a sucessive post.


    > The thing is,
    > that this process ALSO iterates over the whole list (although only for a
    > fast reference copy).
    >
    > I had just made a suggestion that is another way to solve the OPs
    > problem, using new Java 1.5 Concurrency API features.
    >
    > The approach to take depends on the size of the list, and the required
    > performance characteristics of the UDP handler.


    Now that you mention it -- yeah, his scenario really calls for a
    blocking queue. Although I had even named a variable in the sample code
    I posted thus, I had focussed my attention too strongly on the iteration
    part.

    Thanks for improving the answers.


    --
    I would like to thank all the fish who have taken part in this post.
    I hope that other fish will follow the example of those who have
    participated, so that, in future, fish all over the world will live
    together in harmony and understanding, and put aside their petty
    differences, cease pursuing and eating each other and live for a
    brighter, better future for all fish, and those who love them.
    Daniele Futtorovic, Jul 19, 2008
    #8
  9. Jimmie Tyrrell

    Daniel Pitts Guest

    Daniele Futtorovic wrote:
    > On 2008-07-20 00:48 +0100, Daniel Pitts allegedly wrote:
    >> And what you did doesn't copy the list either, it just wraps it. To
    >> copy it you'd actually have to use new ArrayList(someQueue);

    >
    > Yes, I'd corrected that in a sucessive post.

    Yes, I noticed that after the fact. I had 300+ posts to get through, and
    thought to reply in the order I read them. :)

    >> The thing is, that this process ALSO iterates over the whole list
    >> (although only for a fast reference copy).
    >>
    >> I had just made a suggestion that is another way to solve the OPs
    >> problem, using new Java 1.5 Concurrency API features.
    >>
    >> The approach to take depends on the size of the list, and the required
    >> performance characteristics of the UDP handler.

    >
    > Now that you mention it -- yeah, his scenario really calls for a
    > blocking queue. Although I had even named a variable in the sample code
    > I posted thus, I had focussed my attention too strongly on the iteration
    > part.


    It happens to have been a problem that I've had to solve several times,
    and each time I had to solve it different ways because of different
    constraints.

    :)
    --
    Daniel Pitts' Tech Blog: <http://virtualinfinity.net/wordpress/>
    Daniel Pitts, Jul 20, 2008
    #9
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Saravanan Rathinavelu

    Iterate through ArrayList using an another ArrayList

    Saravanan Rathinavelu, Aug 16, 2003, in forum: ASP .Net
    Replies:
    3
    Views:
    2,727
    Natty Gur
    Aug 19, 2003
  2. Kaidi
    Replies:
    4
    Views:
    2,334
    Kaidi
    Jan 3, 2004
  3. Eren Aykin
    Replies:
    2
    Views:
    4,689
    Eren Aykin
    Apr 23, 2006
  4. xz
    Replies:
    16
    Views:
    2,344
  5. Philipp
    Replies:
    6
    Views:
    902
    Arne Vajhøj
    May 28, 2008
Loading...

Share This Page