Regular expression to exclude lines?

S

Shannon Jacobs

Sorry to ask what is surely a trivial question. Also sorry that I don't have
my current code version on hand, but... Anyway, must be some problem with
trying to do the negative. It seems like I get into these ruts each time I
try to deal with regular expressions.

All I'm trying to do is delete the lines which don't contain a particular
string. Actually a filter to edit a log file. I can find and replace a thing
with null, but can't figure out how to find the lines which do not contain
the thing.

Going further, I want to generalize and use a JavaScript variable containing
the decision string, but first I need to worry about the not-within-a-line
problem.
 
T

Thomas 'PointedEars' Lahn

Shannon said:
Sorry to ask what is surely a trivial question.

Hm, I don't think it is this trivial.
All I'm trying to do is delete the lines which don't contain a
particular string. Actually a filter to edit a log file. I can
find and replace a thing with null, but can't figure out how to
find the lines which do not contain the thing.

Here's a quickhack that filters out of three lines the one that
does not contain the word `line':

alert("this is a line\nthis is a\nthis is a
line".match(/\n*[^\n]*\n*([^\n]*[^l][^i][^n][^e][^\n]*\n)*[^\n]*\n*/)[1])

But there must be a better a better way, IIRC there is something
called `negative lookahead', supported from JavaScript 1.5 on,
which I have yet not worked with.


PointedEars
 
L

Lasse Reichstein Nielsen

Thomas 'PointedEars' Lahn said:
Hm, I don't think it is this trivial.

Neither do I. Negative matches in regular expressions rarely are.
Here's a quickhack that filters out of three lines the one that
does not contain the word `line':

alert("this is a line\nthis is a\nthis is a
line".match(/\n*[^\n]*\n*([^\n]*[^l][^i][^n][^e][^\n]*\n)*[^\n]*\n*/)[1])

That's purely accidental. If you add a line in front, e.g.,
"bad thing\nthis is a line\nthis is a\n this is a line", it matches
the string containing the second and third line.
But there must be a better a better way, IIRC there is something
called `negative lookahead', supported from JavaScript 1.5 on,
which I have yet not worked with.

Negative lookahead might be an easier way to do it.

The hard way:

/^([^l\n]*(l[^i]|li[^n]|lin[^e]))*([^l\n])*$/m
(any "l" is not followed by "ine")

With negative lookahead:
/^([^l\n]*l(?!ine))*[^l\n]*$/m

The "m" at the end makes "^" and "$" match beginning/end of line.

These regexps only check for the letters "line", not whether they
occur as a word. To do that, one must check for word boundaries around it:

Hard:
/^([^l\n]*(\bl([^i]|i[^n]|in[^e]|ine\B)|\Bl))*[^l\n]*$/m
Easy:
/^([^l\n]*(\bl(?!ine\b)|\Bl))*[^l\n]*$/m

Any "l" right after a word boundary is not followed by ine+word boundary.

To test this regexp, try:
---
var regexp = /^([^l\n]*(\bl(?!ine\b)|\Bl))*[^l\n]*$/mg ;
var lines = "nonline\nline\nlinefeed\nwith line in the middle\n"+
"no l-word here\n\nprevious l-word was empty\nand ending in line";
var dellines = lines.replace(regexp,"---DELETED---");
alert(lines);
alert(dellines);
---


A longer explanation of:
/^([^l\n]*(\bl(?!ine\b)|\Bl))*[^l\n]*$/m
^ beginning of line
^ some non-l/non-newlines
^ either wordboundary + l not followed by "ine"+wordboundary
^or l not after word boundary
^any number of times
^ and then some non-l/non-newlines again.

Good luck:)
/L
 
E

Evertjan.

Lasse Reichstein Nielsen wrote on 24 nov 2003 in comp.lang.javascript:
Negative lookahead might be an easier way to do it.

What about this non greedy "*?" form:

<script>

function replLine(x,t){
t+="\n"
var re = new RegExp("[^\n]*?"+x+"[^\n]*\n","g");
t = t.replace(re ,"")
return t.replace(/\n$/,"")
}

tx="bad thing\nthis is a line\nthis is a\n this is a line"

alert(replLine("thing",tx))
alert(replLine("line",tx))

</script>
 
E

Evertjan.

Evertjan. wrote on 24 nov 2003 in comp.lang.javascript:
Lasse Reichstein Nielsen wrote on 24 nov 2003 in comp.lang.javascript:
Negative lookahead might be an easier way to do it.

What about this non greedy "*?" form:

<script>

function replLine(x,t){
t+="\n"
var re = new RegExp("[^\n]*?"+x+"[^\n]*\n","g");
t = t.replace(re ,"")
return t.replace(/\n$/,"")
}

tx="bad thing\nthis is a line\nthis is a\n this is a line"

alert(replLine("thing",tx))
alert(replLine("line",tx))

</script>

"All I'm trying to do is delete the lines which don't contain a
particular string. "

Wow, I missed the "n't"

I will try again later.
 
E

Evertjan.

Evertjan. wrote on 24 nov 2003 in comp.lang.javascript:
"All I'm trying to do is delete the lines which don't contain a
particular string. "

Wow, I missed the "n't"

I will try again later.

This better?

<script>

function replLine(x,t){
var re = new RegExp(x,"");
t+="\n"
t = t.replace(
/.*?\n/g,
function($0,$1,$2)
{return (!re.test($0))?$0:""}
)
return t.replace(/\n$/,"")
}

tx="bad thing\nthis is a line\nthis is a\n this is a line"

alert(replLine("thing",tx))
alert(replLine("line",tx))

</script>
 
E

Evertjan.

Evertjan. wrote on 24 nov 2003 in comp.lang.javascript:
Evertjan. wrote on 24 nov 2003 in comp.lang.javascript:


This better?

<script>

function replLine(x,t){
var re = new RegExp(x,"");
t+="\n"
t = t.replace(
/.*?\n/g,
function($0,$1,$2)
{return (!re.test($0))?$0:""}
)
return t.replace(/\n$/,"")
}

tx="bad thing\nthis is a line\nthis is a\n this is a line"

alert(replLine("thing",tx))
alert(replLine("line",tx))

</script>

Monologue follows.

Damn, forgot to remove the "!"

<script>

function replLine(x,t){
var re = new RegExp(x,"");
t+="\n"
t = t.replace(
/.*?\n/g,
function($0,$1,$2)
{return (re.test($0))?$0:""}
)
return t.replace(/\n$/,"")
}

tx="bad thing\nthis is a line\nthis is a\n this is a line"

alert(replLine("thing",tx))
alert(replLine("line",tx))

</script>
 
D

Dr John Stockton

JRS: In article <[email protected]>, seen in
Thomas 'PointedEars' Lahn
Shannon said:
Sorry to ask what is surely a trivial question.

Hm, I don't think it is this trivial.
All I'm trying to do is delete the lines which don't contain a
particular string. Actually a filter to edit a log file. I can
find and replace a thing with null, but can't figure out how to
find the lines which do not contain the thing.

Here's a quickhack that filters out of three lines the one that
does not contain the word `line':

alert("this is a line\nthis is a\nthis is a
line".match(/\n*[^\n]*\n*([^\n]*[^l][^i][^n][^e][^\n]*\n)*[^\n]*\n*/)[1])

But there must be a better a better way, IIRC there is something
called `negative lookahead', supported from JavaScript 1.5 on,
which I have yet not worked with.


AIUI, the OP wants a file which is the previous file minus those lines
which do not contain the string. That code, after broken-string
correction, pops up a box showing the first unwanted line.

Javascript "alone" is not capable of file handling, AFAICS.

If the OP can read and write the file line by line, controlled by
javascript, and apply script to each line, then it is only necessary to
do (pseudo-code follows)

while not EoF(FI) do begin Readln(FI, S) ; // pascal
if ( ! /«string»/.test(S) ) continue // javascript
Writeln(FO, S) end ; // pascal




The OP has MSOE, which suggests Windows. If the job is to be run in
DOS, Windows, or UNIX, then the task is trivial using MiniTrue, which
IMHO is a most valuable tool. Example :

mtr -no~ jt.htm - e

will put, on standard output, all those lines of jt.htm which do not
contain the letter e. A RegExp can be used for the search, in place
of e. There may be a way of doing it without using standard output.
 
L

Lasse Reichstein Nielsen

Evertjan. said:
Evertjan. wrote on 24 nov 2003 in comp.lang.javascript:
Monologue follows.

Damn, forgot to remove the "!"
function replLine(x,t){
var re = new RegExp(x,"");
t+="\n"
t = t.replace(
/.*?\n/g,
function($0,$1,$2)
{return (re.test($0))?$0:""}
)
return t.replace(/\n$/,"")
}

This first splits the string into lines, and then replaces each line
based on a second test.
It should work (and seems to).

I don't think you need a non-greedy match (.*?) since . doesn't match
a newline character.


Maybe you can get around adding the extra "\n" by using multiline
matching: /^.*$/gm,
It doesn't remove the newlines in the string though. None of my
attempts have done that so far.

This method uses several regexp matches, not just one (which is
sometimes the better approach :), but the first is really just
splitting into lines. You can use the split method for that.

How about this:

// returns new array containing only those elements that match re
Array.prototype.filter = function filter(re) {
var res = [];
for (var i=0;i<this.length;i++) {
if (re.test(this)) {res.push(this);}
}
return res;
}

var tx="bad thing\nthis is a line\nthis is a\n this is a line";
alert(tx.split("\n").filter(/line/).join("\n"));

Sadly, adding properties to Array.prototype means that you can't
(easily) use
for (var i in this)
to iterate through a sparse array. The filter method is enumerable,
so it is also included.

/L
 
E

Evertjan.

Lasse Reichstein Nielsen wrote on 24 nov 2003 in comp.lang.javascript:
I don't think you need a non-greedy match (.*?) since . doesn't match
a newline character.


True !
 
M

Mark Szlazak

Try this to exclude lines that don't have "something" in them:

rx = /^(?:(?!\bsomething\b).)*$/gm;
output = input.replace(rx,'');
 
D

Dr John Stockton

JRS: In article <[email protected]>, seen
All I'm trying to do is delete the lines which don't contain a particular
string. Actually a filter to edit a log file. I can find and replace a thing
with null, but can't figure out how to find the lines which do not contain
the thing.

Of course, for a *particular* string, not requiring a RegExp, DOS batch
provides the answer, and there must surely be a UNIX equivalent.

find "mystring" < old.log > new.log

It seems likely that someone has written a version of DOS find that
accepts RegExps; given RegExp code in library form, the job seems
trivial. UNIX has grep, which should do; and there are ports of grep to
DOS & Windows. Also, WSH has file I/O and RegExps, AIUI.

The important thing seems to be to not bother with a RegExp
substitution, but to work line-by-line and do a RegExp (or other) test
of acceptability.
 
T

Thomas 'PointedEars' Lahn

Dr said:
Of course, for a *particular* string, not requiring a RegExp, DOS batch
provides the answer, and there must surely be a UNIX equivalent.

find "mystring" < old.log > new.log

grep -v 'mystring' old.log >new.log 2>&1

The single quotes are only required if special shell expressions are
used but not escaped. 2>&1 captures (error) messages in new.log, too.
If you do not want that, leave it out.


PointedEars
 
D

Dr John Stockton

JRS: In article <[email protected]>, seen in
Thomas 'PointedEars' Lahn
grep -v 'mystring' old.log >new.log 2>&1

The single quotes are only required if special shell expressions are
used but not escaped. 2>&1 captures (error) messages in new.log, too.
If you do not want that, leave it out.

I'm not aware of 2>&1 being valid in either DOS or Win98, and GREP is
not part of those systems but must be imported.

Why did you cut the part where I wrote "UNIX has grep, which should do;
and there are ports of grep to DOS & Windows." ?

IMHO, MiniTrue is more useful than GREP and SED; see my reply to your
earlier post.
 
T

Thomas 'PointedEars' Lahn

Dr said:
I'm not aware of 2>&1 being valid in either DOS or Win98,

It is only available in Cmd.exe of Windows NT-based systems and
Unices, generally saying in POSIX-compatible shells, of course.
and GREP is not part of those systems but must be imported.

Of course. My posting was only regarding "there must
surely be a UNIX equivalent". Here you are :)
Why did you cut the part where I wrote "UNIX has grep, which should do;
and there are ports of grep to DOS & Windows." ?

Just oversaw it.
IMHO, MiniTrue is more useful than GREP and SED; see my reply to your
earlier post.

MiniTrue is not part of a basic installation of Unices,
though. (Instead, mtr refers to Matt's Traceroute.)


F'up2 poster

PointedEars
 
D

Dr John Stockton

JRS: In article <[email protected]>, seen in
Thomas 'PointedEars' Lahn
It is only available in Cmd.exe of Windows NT-based systems and
Unices, generally saying in POSIX-compatible shells, of course.

There is no reason to assume that the OP is aware of that; or even of
that part of that that is applicable to the system in question. a
plausible but inapplicable or incorrect "solution" is worse than
useless".

MiniTrue is not part of a basic installation of Unices,
though. (Instead, mtr refers to Matt's Traceroute.)

Indeed; nor of DOS; which is sufficiently clearly indicated by my
signature to that article.
 
T

Thomas 'PointedEars' Lahn

Dr said:
There is no reason to assume that the OP is aware of that;

There is no reason that she is interested in it, either.
or even of that part of that that is applicable to the system in
question.

The system in question is unknown.
a plausible but inapplicable or incorrect "solution" is worse than
useless".

This is a JavaScript newsgroup, not a shell script newsgroup. As the
OP has not provided the system on which solutions are supposed to work,
she will test if the provided solutions will work on her system.


PointedEars
 
S

Shannon Jacobs

Thomas said:
Dr John Stockton wrote:

There is no reason that she is interested in it, either.
<snip>

Return of the original poster... Sorry I've been rather busy and haven't
been able to follow this very interesting thread more closely. (The OP is
male, by the way.) However, judging by the complexity of the discussion, it
seems that there was some reason for my original perplexity, though I
thought it a trivial notion.

First let me try to clarify what I'm doing. I know that JavaScript has no
access to the file system. The files are to be handled directly by the user
of the utility. In the Windows environment, this is trivial with ^A, ^C, and
^V. I didn't mention that part because it's almost mindless now (for me).
The actual steps for the file part are:

Open the converter JavaScript form, then open the target file, ^A, ^C, click
in the form, ^V, click on the convert button of the form, ^A, ^C, click back
in the original file, ^A, ^V, and save the file. Done. (If anyone is curious
and the results of these steps are not obvious enough, I can explain.)

Now to clarify the JavaScript part. This example is from an existing utility
that converts raw HTML into JavaScript. The variable HtmlText is the body of
the file from an input field in the form. The critical function is:

function jsFromHtml(HtmlText) {
HtmlText = HtmlText.replace(/\"/g,"\\\"");
HtmlText = HtmlText.replace(/[\r\n]+/g,"\");\r\ndocument.writeln(\"");
return "document.writeln(\"" + HtmlText + "\");";
}

I know this is rather ugly code, and I'd also be interested in improvements,
or even a completely different approach. My JavaScript skills are obviously
rather limited, but this was adequate for my purposes at the time. Since
it's probably not IOttMCO, I'll explain what it does. In the first
executable line, the regular expression escapes all of the double quotes in
the original HTML. In the next line, all of the embedded line breaks are
replaced with the end and start of document.writeln statements, and then the
last line puts one more start and end around the entire thing. The result is
a block of JavaScript code which outputs the arbitrary HTML input. You stick
that into a JavaScript function to create that block of HTML under program
control wherever it is required. (I was especially unhappy with my treatment
of line breaks, and believe this is not a properly general method, though it
works.)

My goal now is to do something similar, but excluding the lines that do not
contain some string. I'm most interested in an elegant solution, though the
discussion so far seems to suggest that there may be no better approach than
parsing the input one line at a time...

An additional wrinkle is that I'd like to generalize a bit by treating the
decision string as a parameter returned in another field of the form.
 
L

Lasse Reichstein Nielsen

Shannon Jacobs said:
First let me try to clarify what I'm doing. I know that JavaScript has no
access to the file system. The files are to be handled directly by the user
of the utility. In the Windows environment, this is trivial with ^A, ^C, and
^V.

I have pages like that too, for colorizing HTML and Javascript :)
Now to clarify the JavaScript part. This example is from an existing utility
that converts raw HTML into JavaScript. The variable HtmlText is the body of
the file from an input field in the form. The critical function is:

function jsFromHtml(HtmlText) {
HtmlText = HtmlText.replace(/\"/g,"\\\"");
HtmlText = HtmlText.replace(/[\r\n]+/g,"\");\r\ndocument.writeln(\"");
return "document.writeln(\"" + HtmlText + "\");";
}
I know this is rather ugly code, and I'd also be interested in improvements,
or even a completely different approach.

I would use split:

function jsFromHtml(HtmlText) { // I would write HTML in all caps :)
var inputLines = HtmlText.split(/[\r\n]+/);
var outputLines = [];
for (var i=0;i<inputLines.length;i++) {
var safeLine = inputLines.replace(/[\\"]/g,"\\$&");
outputLines = "document.writeln(\"" + safeLine + "\");" ;
}
return outputLine.join("\n");
}

(I put a backslash in front of both double quotes and backslashes,
since neither can occour alone in a string. If there are other
characters that makes no sense in a string, they should be
handled as well. Examples could be \t or \b).
My JavaScript skills are obviously rather limited,

Not obviously. It works, and it's something I could find myself doing if
I didn't have split available. It's possibly even faster to change all
the quotes from the beginning instead of doing one replace per line.
but this was adequate for my purposes at the time. Since it's
probably not IOttMCO,

You lost me there :) IOttMCO?
I'll explain what it does.

It's fairly easy to read, as long as you can see what's inside a string
and what's not :)
My goal now is to do something similar, but excluding the lines that do not
contain some string. I'm most interested in an elegant solution, though the
discussion so far seems to suggest that there may be no better approach than
parsing the input one line at a time...

Nothing wrong with one line at a time. If you use the code I showed above,
all you need is to wrap the content of the for loop in an if statement:

if (!/badWord/.test(inputLines)) {
... add to output ...
}

or

if (inputLines.indexOf("badWord") == -1) {
... add to output ...
}

An additional wrinkle is that I'd like to generalize a bit by treating the
decision string as a parameter returned in another field of the form.

var testRE = RegExp(form.elements['otherField'].value);
if (! testRE.test(inputLines)) {
...
}

(To avoid problems or crashes, you might want to screen the other field's
values for characters that are meaningful in regular expressions)
or

var testWord = form.elements['otherField'].value;
if (inputLines.indexOf(testWord)==-1) {
... add to output ...
}


Good luck
/L
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top