Writing forum reply extension - regex

[ivanjaros]

I am writing a block-typed extension to quote forum posts in replies.
Specifically I am looking at [quote user=123 name=John Doe timestamp=123456]foo bar baz lorem ipsum etc[/quote]

which should result in something like:

John Doe said on 1st April, 2030:
foo bar baz lorem ipsum etc

plus some html wrappers.

The extension must also work with child tokens so quote can embed another quote and so on.

The base is simple:

const quote = {
  name: 'quote',
  level: 'block',
  start(src) {
    const m = src.match(/^\[quote\](\S*?)\[\/quote\]/) <-- this works for inline types but does it for blocks as well?
    return m ? m.index : null
  },
  tokenizer(src, tokens) {
    const rule = /^\[quote\](\S*?)\[\/quote\]/;
    const match = rule.exec(src);
    if (match) {
      return {
        type: "quote",
        raw: match[0],
        user_id: "", @todo 
        user_name: "", @todo 
        message: "", @todo 
        timestamp: 0, @todo 
      }
    }
  },
  renderer(token) {
    return '<script rel="quote" type="application/json">' + JSON.stringify({uid: token.user_id, name: token.user_name, timestamp: token.timestamp, message: token.message}) + '</script>'
  }
};

I am using script render because in case the renderer is not configured to recognize the extension, the html sanitizer will simply remove it altogether from the final result.

I think I can figure out the children rendering. I have one component that does it(this.lexer.inline(token.text, token.tokens); so it should be the same thing. Which leaves the pattern matching to find out the user id, name, timestamp. Which are optional.

I would guess I should do sub-pattern matching for the [quote] and simply look for the individual attributes inside? Though am not sure how to handle space in the name, for example. Maybe using quotes would make more sense, like [quote name="John Doe"]?

But am also not sure how properly render the children in case of quote containing another quote so the regex works as it it supposed to...

Anyone done block extension like this and can help out?

[UziTech]

You can use something like attributes-parser to parse the attributes.

Here is a working example:

import { Marked } from 'marked';
import parseAttrs from 'attributes-parser';

const marked = new Marked();

const quoteRule = /^\[quote([^\]]*)\]([^]*?)\[\/quote\]/;
const quoteStartRule = new RegExp(quoteRule.source, 'm');
const quote = {
  name: 'quote',
  level: 'block',
  start(src) {
    const m = src.match(quoteStartRule);
    return m ? m.index : null;
  },
  tokenizer(src, tokens) {
    const match = quoteRule.exec(src);
    if (match) {
      return {
        type: 'quote',
        raw: match[0],
        attrs: parseAttrs(match[1]),
        text: match[2],
        tokens: this.lexer.blockTokens(match[2]),
      };
    }
  },
  renderer(token) {
    console.log(token);
    return '<script rel="quote" type="application/json">' + JSON.stringify(token.attrs) + '</script>\n';
  },
};

marked.use({ extensions: [quote] });

console.log(marked.parse(`
test
[quote user=123 quote="\\\"" name="John Doe" timestamp=123456]foo bar baz lorem ipsum etc[/quote]
`));

nesting will be much harder because when the regexp matches [/quote] you will have to have some way of knowing it is not the end of an inner quote.

[ivanjaros]

From discussions it seems best way will be to use regex to find beginning of quote but then manually lookahead and find next quote opening or closing in order to determine where the actual closing tag is. Then take the content and use lexer to tokenize it further.

[UziTech]

yes you will also have to deal with the case where there is a nested quote but no ending [/quote] for the initial quote. What you want to do in that case is a matter of preference. Either end the quote at the end of the markdown or consider that not a quote if the initial quote doesn't have an ending [/quote] or the initial and nested quote end at the only [/quote].

[ivanjaros]

...ah, it's the next day, i have a clear head, i think i will just add a new tag, like [user=123]john doe[/user] and prepend the quote with it, rather than try to make the original idea work. even here on github, it does not use anything more than a markdown blockquote. the main idea was merely to be able to notify the user about being mentioned, so this will do just that without having to alter anything related to quotes themselves.