Double match - substitute pattern on subsequent line if prevous line matches another pattern?

Question

I'm trying to replace VEVENT to VTODO entries in an .ics file if it matches current date on another line (it was exported incorrectly):

BEGIN:VCALENDAR
BEGIN:VEVENT
DTSTART:20220340T140000
END:VEVENT
BEGIN:VEVENT
DTSTART:20230620T193700
END:VEVENT
BEGIN:VEVENT
DTSTART:20210210T193800
END:VEVENT
END:VCALENDAR

The second VEVENT entry has current time so it should become:

BEGIN:VTODO
DTSTART:20230620T193700
END:VTODO

There are more entries between BEGIN:VEVENT and END:VEVENT lines, I've redacted them for clarity.

I've tried this with sed, but the ranges pick the first occurrence of VEVENT in the entire file, not first occurrence after (or before) the matched pattern, so it replaces all of them.

sed -i "/BEGIN:VEVENT/,/DTSTART:$(date +%Y%m%dT%H%M)/{s/VEVENT/VTODO/}" org.ics

I was trying to adapt it to another question here, which I thought was relevant: Find a string and replace another string after the first is found

sed -n "/DTSTART:$(date +%Y%m%dT%H%M)/,${/END:VEVENT/{x//{x b}g s/VEVENT/VTODO/}}" org.ics

but it didn't work at all: sed: -e expression #1, char 25: unexpected ,'`

Is this helpful? `sed 'H;/BEGIN:VEVENT/h;/END:VEVENT/!d;x;/DTSTART:20230620T193700/s/VEVENT/VTODO/g' org.ics` ? — Valentin Bajrami, Jun 20 '23 at 20:53
oh wow, yes it did work. That's amazing! I've spent so much time trying to solve this. Could you also recommend me any way to insert the `$(date +%Y%m%dT%H%M)` command? When I changed single quotes for double it didn't seem to work anymore and exclamation mark character was being expanded. I was planning to run it as a cron job that amends the export file on each run, so dynamic date command would be useful. — Daniel Krajnik, Jun 20 '23 at 21:08
Do you think that you could post it as an answer and explain a bit more what's the design? I'd be curious to learn it (haven't used H or x in sed before) — Daniel Krajnik, Jun 20 '23 at 21:10
Sure! `sed 'H;/BEGIN:VEVENT/h;/END:VEVENT/!d;x;/DTSTART:'"$(date +%Y%m%dT%H%M)"'/s/VEVENT/VTODO/g' org.ics` — Valentin Bajrami, Jun 20 '23 at 21:11
that is brilliant, so simple. Thank you again. Yes, if you would like to explain it as well that would be very intersesting. I can see that there are three initial addresses and somehow it allows you to perform substitution by checking the third address, but I'm completely lost on how that happens and what "H" "h" and "x" are — Daniel Krajnik, Jun 20 '23 at 21:16
Sorry, I've just noticed it. The only minor problem is that it removes the BEGIN:VCALENDAR, END:VCALENDAR and other lines outside the VEVENT entries (the rest is all correct). It's not a big deal, because these VCALENDAR entries are always the same, so one can just copy it in the shell script before sed runs, but if you knew what might be causing please let me know. — Daniel Krajnik, Jun 20 '23 at 21:31
Perfect, thank you again. So "1n" prints everything before the first address? This truly is high level `sed wizardry`! — Daniel Krajnik, Jun 20 '23 at 22:44
Regarding `There are more entries between BEGIN:VEVENT and END:VEVENT lines, I've redacted them for clarity.` - if any of those might contain BEGIN, END, VEVENT, DSTART, etc. as [sub]strings (e.g. maybe there's some kind of description field about the event that might say things including typos like "DESCRIPTION:This is the best VEVENT we've had. In the BEGINNING we could DSTART to open presents but it ENDs too soon") then you should include that in your sample input/output as most, if not all, of the posted solutions will fail given some of those strings being present in different locations. — Ed Morton, Jun 21 '23 at 12:08
@EdMorton that's a good point, haven't thought about it. If I knew how complex it gets I would probably have used python's icalendar parser instead. — Daniel Krajnik, Jun 21 '23 at 17:20
With any pattern matching, it's always much easier to match the text you want than it is to not match text you don't want so you have to think long and hard about what the potential input might look like before coming up with a solution as only considering the sunny day input usually leads to shooting yourself in the foot later. — Ed Morton, Jun 21 '23 at 19:58

Valentin Bajrami · Accepted Answer · 2023-06-20T22:07:32.513

So the following should work:

sed 'H;/BEGIN:VEVENT/h;/END:VEVENT/!d;x;/DTSTART:'"$(date +%Y%m%dT%H%M)"'/s/VEVENT/VTODO/g' org.ics

Explanation:

H: Appends to the hold space which creates kind of a buffer (space), then we do pattern matching

/BEGIN:VEVENT/h and store it in the hold space, so now we run another pattern matching

/END:VEVENT/!d and if the pattern doesn't match, then delete it.

x; Exchanges hold space with pattern space so right now we have the lines we need in the pattern space

Finally, substitute the line DTSTART... if it matches the date. So s/.../.../g is executed only if there is a match.

/DTSTART:'"$(date +%Y%m%dT%H%M)"/s/VEVENT/VTODO/g

Update

sed '1n;H;/BEGIN:VEVENT/h;/END:VEVENT/!d;x;/DTSTART:'"$(date +%Y%m%dT193700)"'/s/VEVENT/VTODO/g' org.ics | sed '$aEND:CALENDAR

This is prefect, thank you. I've had to add a few things for it to handle correctly beginning and end of the file and be compatible with systemd. It's too much a stackexchange comment, but it seems to be finally working - events and todos are correctly separated in thunderbird (I assume that google calendar and other clients will follow suit). Thanks again! — Daniel Krajnik, Jun 21 '23 at 00:44

score 1 · Answer 2 · answered Jun 21 '23 at 12:31

Robustly using any awk in any shell on every Unix box:

$ cat tst.sh
#!/usr/bin/env bash

awk -v now="$(date +'%Y%m%dT%H%M')" '
    $0 == "BEGIN:VEVENT" {
        eventType = 1
        event = $0
        next
    }
    eventType {
        event = event ORS $0
        if ( $0 == ("DTSTART:" now) ) {
            eventType = 2
        }
        if ( $0 == "END:VEVENT" ) {
            if ( eventType == 2 ) {
                sub(/^BEGIN:VEVENT/,"BEGIN:VTODO",event)
                sub(/END:VEVENT$/,"END:VTODO",event)
            }
            print event
        }
        next
    }
    { print }
' "${@:--}"

$ ./tst.sh file
BEGIN:VCALENDAR
BEGIN:VEVENT
DTSTART:20220340T140000
END:VEVENT
BEGIN:VTODO
DTSTART:20230620T193700
END:VTODO
BEGIN:VEVENT
DTSTART:20210210T193800
END:VEVENT

The only way that would fail would be if you could have some other section in your input that can contain lines that match any of the BEGIN:, END:, or DSTART: lines but I doubt if that could be valid in your input - you'd have to show us how it's specified if it is.

The important thing about the above solution is that it's comparing whole literal lines to the target strings, not just doing partial matches on wherever they might occur in the input.

score 0 · Answer 3 · answered Jun 20 '23 at 19:50

0

It is not solving, but the hint only:

sed s/'$'/':'/ org.ics |awk 'BEGIN{RS="BEGIN";FS=":";CD="20220340"} NR>1 {AA[1]="BEGIN";for(i=2;i<=NF;i++){AA[i]=$i;if($i =="DTSTART" && $(i+1) ~ CD ){AA[2]="VTODO"}};for(i in AA){printf AA[i]":"};print ""}'

Where sed command adds : (colon) at the end to each line, RS="BEGIN" sets the RowSeparator to keyword BEGIN (that's why it dismisses!), FS=":" sets the FieldSeparator to ":" . In each "row" (from the keyword BEGIN to the next BEGIN, better said "record") the fields are copied into the array members AA[i] starting with i=2, because the AA[1] has to be the dismissed BEGIN. While checking the keyword DTSTART and current date, and changing the VEVENT to VTODO if necessary. After the line is checked, the array AA[] is printed and with : as separator.

This is the hint only, I did not test it. You have to debug and tune up by yourself. Namely you may clear the ending : if it is unwanted.

answered Jun 20 '23 at 19:50

schweik

1,160
8
16

Thank you, this makes sense, but I am not that good with awk and the other answer was a bit shorter. If it turns out that there will be other entries after VEVENT that would invalidate it I may revisit this awk solution. – Daniel Krajnik Jun 20 '23 at 22:47
You never need `sed` when you're using `awk`. If you want a colon at the end of each line then just do `$0=$0":"` or `sub(/$/,":")` in awk. You should mention that awk script requires an awk that supports multi-char RS, e.g. GNU awk but would fail with a POSIX-only awk. – Ed Morton Jun 21 '23 at 12:00
It was not clearly set how many fields are between BEGIN/END and if they keep the same order. The code can be much more simplified if the records keep the fixed structure. TXR looks very impressive and the `sed` solution is also nice. I appreciate everybody with deep knowledge of some program(ming language), but the life is too short to learn more than two/three of them. But the basic skill is the art of analyzing a problem and finding a way. The gu-ru, who can find the most effective way, can speak the specific language as native, and some of them has dreams in that language. – schweik Jun 21 '23 at 16:30

Kaz · Answer 4 · 2023-06-21T14:25:25.253

We can have a solution like this in TXR:

@(repeat)
@  (cases)
BEGIN:VEVENT
DTSTART:@{date}T@{time}
END:VEVENT
@    (output)
BEGIN:VTODO
DTSTART:@{date}T@{time}
END:VTODO
@    (end)
@  (or)
@line
@    (do (put-line line))
@  (end)
@(end)

This, by itself, will rewrite all the entries to VTODO:

$ txr  cal.txr cal.ics
BEGIN:VCALENDAR
BEGIN:VTODO
DTSTART:20220340T140000
END:VTODO
BEGIN:VTODO
DTSTART:20230620T193700
END:VTODO
BEGIN:VTODO
DTSTART:20210210T193800
END:VTODO
END:VCALENDAR

We can bind the date and time variables from the command line though:

$ txr -Ddate=20230620 cal.txr cal.ics
BEGIN:VCALENDAR
BEGIN:VEVENT
DTSTART:20220340T140000
END:VEVENT
BEGIN:VTODO
DTSTART:20230620T193700
END:VTODO
BEGIN:VEVENT
DTSTART:20210210T193800
END:VEVENT
END:VCALENDAR

Now that only matches and rewrites the second entry.

Pretty version:

Thanks, I haven't heard of TXR before. Looks powerful. I think the link from the first paragraph should be https://www.nongnu.org/txr/ ? — Daniel Krajnik, Jun 21 '23 at 09:38

Double match - substitute pattern on subsequent line if prevous line matches another pattern?

4 Answers4