Can ruby do this?

[no.top.post]

Consider files represented thus:
X123456abcd|Y123456efg|Z123456hijk|

Where:
X,Y,Z are <header/description lines>
| is the line which terminates/separates each of the [3 in this case]
sections of the file.
1,2...6 are lines that are repeated [hence redundant] in each section.
a,b,....k are the lines of CONTENTS that you want to end up with.

The algorithm may be something like this:
IF any 3 or more consecutive lines of the 1st section
are repeated in any of the following sections
THEN delete these <repeats>.

So you want to end up with:
X123456abcd|Yefg|Zhijk|

PS. this problem is a further refinement of
< with MaxLineLen= 77 ; lynx-fetch the filed-list-of-URLs and
append them to File2, each with its header & separator. >

You want to delete the repeated garbage.
Like if you d/l 8 pages of a 'publication' each page will have the same
47-line 'header'.

So you want to discard the redundant packaging.

== TIA.

 

[Robert Klemme]

Consider files represented thus:
X123456abcd|Y123456efg|Z123456hijk|

Where:
X,Y,Z are<header/description lines>
| is the line which terminates/separates each of the [3 in this case]
sections of the file.
1,2...6 are lines that are repeated [hence redundant] in each section.
a,b,....k are the lines of CONTENTS that you want to end up with.

The algorithm may be something like this:
IF any 3 or more consecutive lines of the 1st section
are repeated in any of the following sections
THEN delete these<repeats>.

So you want to end up with:
X123456abcd|Yefg|Zhijk|

PS. this problem is a further refinement of
< with MaxLineLen= 77 ; lynx-fetch the filed-list-of-URLs and
append them to File2, each with its header& separator.>

You want to delete the repeated garbage.
Like if you d/l 8 pages of a 'publication' each page will have the same
47-line 'header'.

So you want to discard the redundant packaging.

Yes.

robert