Markdown parser

[emeralit]

The field message accepts markdown text and Comma renders it correctly.

[Dieterbe]

we should not include binaries in the source repo

[Dieterbe]

you can have such a message in your fork, but this should not go into the upstream, e.g. not in this pr

[Dieterbe]

your comments.html is in italian...
perhaps we should have an "examples" directory with different versions of comments.html, e.g. this could be examples/comments-it.html ? and i can add my copy too (english version)

[Dieterbe]

what does md.Convert() do exactly? does it parse markdown into html?
i wonder when is the best time to convert md to html... we basically have 3 options:

1. in the backend, when receiving the form, before saving to the file
2. in the backend, after reading the file, before shipping to frontend
3. in the frontend, after receiving from backend, before adding them to DOM

it looks you do 1, the issue with this, if you later want to tweak the translation from md to html, this is not possible anymore after files have been saved.

seems 2 or 3 would be better.

[Dieterbe]

huh? why? this would break gravatar, no?

[Dieterbe]

gofmt

[emeralit]

Hi,
thank you for your comments. I am studying a more safe alternative.

[emeralit]

Hello,
after taking note of your comments and discussing them in our small group of friends, we have found a possible solution. Comma is left as you conceived it. To render the markdown code, we used DOMPurify and Marked. The first one sanitizes any malicious code that might be inserted into the markdown, the latter render the text.
What do you think about this solution?

[Dieterbe]

marked looks interesting for client side rendering. regarding the validation, what are the concerns exactly?

[emeralit]

Our main concerns are:
Security Concerns: Client-side rendering can potentially expose sensitive information and logic, making it vulnerable to injection attacks and other security vulnerabilities.
Data Integrity: Ensuring the accuracy and integrity of data validation on the client side is crucial to prevent erroneous or malicious data from being processed by the server.
We are helping a friend to manage a static blog transition from the late Muut to another system. Comma is suitable for the scope, but someone wants markdown, someone wants modify comments after their publication, someone wants rule the world and so on. But we like a minimal approach to a self-hosted service without a lot of complexity.

[Dieterbe]

oh that's a good point.
i just tried posting a comment like:

<a href="http://dns.be">dns</a>
<h1>test</h1>
    <script>
        let d = new Date();
        alert("Today's date is " + d);
    </script>

the link and h1 work, the script doesn't run for some reason. not sure why. looking at the code, it doesn't look like comma has anything special that would prevent running or loading external scripts, but maybe the browser does this automatically because the content is loaded via ajax.
i always liked the idea that people would be able to use some html markup for bolding, hyperlinking, etc from their comments, but obviously it makes sense to prevent most of everything else (scripts, headlines, etc...)

I suppose this is an issue with both the current system (which allows all html tags), as well as new markdown support (which, I believe, lets you add any arbitrary html code to the markdown as well)

Perhaps the ideal solution would be to not just make markdown an optional feature for some comments, but rather force every comment through a markdown layer, except one that blocks (or escapes) html tags, or at least "the bad ones". this way people could still write links, bold, italics, lists etc (all the useful things) but not able to do anything bad..

[emeralit]

Marked alone is not safe. We added DOMPurify to block code, but the script sanitizes inline code and blockcode as well.

[emeralit]

This is our work in progress on comments.html<div id="comments-container">  <!-- javascript will add comments here. loaded from backend and/or submitted successfully --></div><h3>Commenta</h3><form id="new-comment-form" action="{{.Site.Params.commentServer}}">  <input type="hidden" name="post" value="{{ .File.BaseFileName }}">  <br /><input type="text" name="name" placeholder="* Nome" required />  <br /><input type="email" name="email" placeholder="* Email (non visibile, usata per Gravatar)" required />  <br /><input type="url" name="url" placeholder="Sito web" />  <br /><input type="text" name="special" placeholder="* Sei un umano?" required />  <br /><textarea rows="10" name="message" placeholder="* Commento" required></textarea>  <button id="new-comment-submit" class="btn-ready" type="submit" value="Send">Invia</button></form><script src="{{ .Site.BaseURL }}js/purify.min.js"></script><script src="{{ .Site.BaseURL }}js/marked.min.js"></script><script language="javascript">  function codeToHTMLentities(mdWithHtml) {      var result = mdWithHtml.replace(/<code>([\s\S]*?)<\/code>/gm, function(a,b){ return "<code>"+toHtmlEntities(b)+"</code>" });            result = result.replace(/^```([\s\S]*?)```$/gm, function(a,b){ return "<pre><code>"+toHtmlEntities(b)+"</code></pre>" });      result = result.replace(/`([^`]+)`/g, function(a,b){ return "<code>"+toHtmlEntities(b)+"</code>" });      return result;  }  function toHtmlEntities(str) { var buf = []; for (var i=str.length-1;i>=0;i--) { buf.unshift(['&#', str[i].charCodeAt(), ';'].join('')); } return buf.join(''); }  var monthNames = ["Gennaio", "Febbraio", "Marzo", "Aprile", "Maggio", "Giugno", "Luglio", "Agosto", "Settembre", "Ottobre", "Novembre", "Dicembre"];  function domReady(fn) {    if (document.readyState === "complete" || document.readyState === "interactive") {      setTimeout(fn, 1);    } else {      document.addEventListener("DOMContentLoaded", fn);    }  }  marked.use({    breaks: true,    gfm: true,  });  var addComment = function (data) {    const container = document.getElementById("comments-container");    var txt = "";    var d = new Date(data.Ts);    var date = d.getDate() + " " + monthNames[d.getMonth()] + " " + d.getFullYear() + " alle " + d.getHours() + ":" + d.getMinutes() + ":" + d.getSeconds();      var msg = marked.parse(data.Message); // Converte il markdown.     txt += '<article id="comment" class="comment">\n';    txt += '\t<img class="commentAvatar" src="https://www.gravatar.com/avatar/' + data.Hash + '?d=mm&s=60">\n';    txt += '\t<div class="commentRight">\n';    txt += '\t\t<p class="commentAuthor">' + ((data.Link == "") ? data.Author : '<a href="' + data.Link + '" class="commentAuthor">' + data.Author + '</a>') + '</p>\n';    txt += '\t\t<time class="entry-date" datetime="' + data.Ts + '">' + date + '</time>\n';    txt += '\t\t<div class="commentMessage">' + msg + '</div>\n';    txt += '\t</div>\n';    txt += '</article>\n';    container.innerHTML += txt;  };  domReady(async function () {    let form = document.getElementById("new-comment-form");    let commentsUrl = "{{.Site.Params.commentServer}}/{{ .File.BaseFileName }}";    form.addEventListener("submit", (event) => {      event.preventDefault();      sendData();    });    const response = await fetch(commentsUrl);    if (response.ok) {      comments = await response.json();      comments.forEach(addComment);    } else {      e = "<div><b>Impossibile caricare i commenti</b></br>" + status + " " + err +        "<br/>Funzione commenti disabilitata";      form.parentNode.insertBefore(document.createElement(e), form);      form.display = "none";    }    async function sendData() {      document.querySelector('#new-comment-submit').textContent = 'Invio in corso...';      const formData = new FormData(form);            // Recupera il testo del commento, lo sanitizza, lo sostituisce nel form      const msg = formData.get('message');      const sanitizedMsg = DOMPurify.sanitize(codeToHTMLentities(msg));      //console.log({msg, sanitizedMsg});      formData.set('message', sanitizedMsg);             try {        const response = await fetch(commentsUrl, {          method: "POST",          body: formData,        });        button = document.querySelector('#new-comment-submit');        if (response.ok) {          button.textContent = 'Inviato!'; // Qui lasciamo pure la diciture Inviato!...          button.className = 'btn-ok';          comment = await response.json();          addComment(comment);          form.reset(); // Clear the form          button.textContent = 'Invia'; // Set the button text back to "Send"          button.className = 'btn-ready'; // Set the button class back to its default state        } else {          button.className = 'btn-err';          const message = await response.text();          button.textContent = 'failed with status ' + response.status + ' ' + message;        }      } catch (e) {        button.textcontent = 'failed with error ' + e;        console.error(e);      }    }  });</script> Commenta InviaInviato da iPhoneIl giorno 12 apr 2024, alle ore 10:18, Dieter Plaetinck ***@***.***> ha scritto: oh that's a good point. i just tried posting a comment like: <a href="http://dns.be">dns</a> <h1>test</h1> <script> let d = new Date(); alert("Today's date is " + d); </script> the link and h1 work, the script doesn't run for some reason. not sure why. looking at the code, it doesn't look like there is anything special that would prevent running or loading external scripts. i always liked the idea that people would be able to use some html markup for bolding, hyperlinking, etc from their comments, but obviously it makes sense to prevent most of everything else (scripts, headlines, etc...) I suppose this is an issue with both the current system (which allows all html tags), as well as new markdown support (which, I believe, lets you add any arbitrary html code to the markdown as well) Perhaps the ideal solution would be to not just make markdown an optional feature for some comments, but rather force every comment through a markdown layer, except one that blocks (or escapes) html tags, or at least "the bad ones". this way people could still write links, bold, italics, lists etc (all the useful things) but not able to do anything bad.. —Reply to this email directly, view it on GitHub, or unsubscribe.You are receiving this because you authored the thread.Message ID: ***@***.***>

[emeralit]

New version of comments.html

<div id="comments-container">
  <!-- javascript will add comments here. loaded from backend and/or submitted successfully -->
</div>

<h3>Commenta</h3>
<form id="new-comment-form" action="{{.Site.Params.commentServer}}">
  <input type="hidden" name="post" value="{{ .File.BaseFileName }}">
  <br /><input type="text" name="name" placeholder="* Nome" required />
  <br /><input type="email" name="email" placeholder="* Email (non visibile, usata per Gravatar)" required />
  <br /><input type="url" name="url" placeholder="Sito web" />
  <br /><input type="text" name="special" placeholder="* Sei un umano?" required />
  <br /><textarea rows="10" name="message" placeholder="* Commento" required></textarea>
  <button id="new-comment-submit" class="btn-ready" type="submit" value="Send">Invia</button>
</form>

<script src="{{ .Site.BaseURL }}js/purify.min.js"></script>

<script src="{{ .Site.BaseURL }}js/marked.min.js"></script>

<script language="javascript">
  function codeToHTMLentities(mdWithHtml) {
      var result = mdWithHtml.replace(/<code>([\s\S]*?)<\/code>/gm, function(a,b){ return "<code>"+toHtmlEntities(b)+"</code>" });
      
      result = result.replace(/^```([\s\S]*?)```$/gm, function(a,b){ return "<pre><code>"+toHtmlEntities(b)+"</code></pre>" });

      result = result.replace(/`([^`]+)`/g, function(a,b){ return "<code>"+toHtmlEntities(b)+"</code>" });

      return result;
  }

  function toHtmlEntities(str) {
    var map = {
      '&': '&amp;',
      '<': '&lt;',
      '>': '&gt;',
      "'": '&#39;',
      '"': '&quot;'
     };
   
     return str.replace(/[&<>"']/g, function(m) { return map[m]; });
	}

  var monthNames = ["Gennaio", "Febbraio", "Marzo", "Aprile", "Maggio", "Giugno", "Luglio", "Agosto", "Settembre", "Ottobre", "Novembre", "Dicembre"];


  function domReady(fn) {
    if (document.readyState === "complete" || document.readyState === "interactive") {
      setTimeout(fn, 1);
    } else {
      document.addEventListener("DOMContentLoaded", fn);
    }
  }

  marked.use({
    breaks: true,
    gfm: true,
  });

  var addComment = function (data) {

    const container = document.getElementById("comments-container");


    var txt = "";
    var d = new Date(data.Ts);
    var date = d.getDate() + " " + monthNames[d.getMonth()] + " " + d.getFullYear() + " alle " + d.getHours() + ":" + d.getMinutes() + ":" + d.getSeconds();

    var msg = DOMPurify.sanitize(marked.parse(data.Message)); // Converte il markdown e lo sanifica.
 
    txt += '<article id="comment" class="comment">\n';
    txt += '\t<img class="commentAvatar" src="https://www.gravatar.com/avatar/' + data.Hash + '?d=mm&s=60">\n';
    txt += '\t<div class="commentRight">\n';
    txt += '\t\t<p class="commentAuthor">' + ((data.Link == "") ? data.Author : '<a href="' + data.Link + '" class="commentAuthor">' + data.Author + '</a>') + '</p>\n';
    txt += '\t\t<time class="entry-date" datetime="' + data.Ts + '">' + date + '</time>\n';
    txt += '\t\t<div class="commentMessage">' + msg + '</div>\n';
    txt += '\t</div>\n';
    txt += '</article>\n';

    container.innerHTML += txt;
  };

  domReady(async function () {
    let form = document.getElementById("new-comment-form");
    let commentsUrl = "{{.Site.Params.commentServer}}/{{ .File.BaseFileName }}";

    form.addEventListener("submit", (event) => {
      event.preventDefault();
      sendData();
    });

    const response = await fetch(commentsUrl);
    if (response.ok) {
      comments = await response.json();
      comments.forEach(addComment);
    } else {
      e = "<div><b>Impossibile caricare i commenti</b></br>" + status + " " + err +
        "<br/>Funzione commenti disabilitata";
      form.parentNode.insertBefore(document.createElement(e), form);
      form.display = "none";
    }

    async function sendData() {
      document.querySelector('#new-comment-submit').textContent = 'Invio in corso...';
      const formData = new FormData(form);
      
      // Recupera il testo del commento, lo sanitizza, lo sostituisce nel form
      const msg = formData.get('message');
      const sanitizedMsg = DOMPurify.sanitize(codeToHTMLentities(msg));
      //console.log({msg, sanitizedMsg});
      formData.set('message', sanitizedMsg);
       
      try {
        const response = await fetch(commentsUrl, {
          method: "POST",
          body: formData,
        });
        button = document.querySelector('#new-comment-submit');
        if (response.ok) {
          button.textContent = 'Inviato!'; // Qui lasciamo pure la diciture Inviato!...
          button.className = 'btn-ok';
          comment = await response.json();
          addComment(comment);
          form.reset(); // Clear the form
          button.textContent = 'Invia'; // Set the button text back to "Send"
          button.className = 'btn-ready'; // Set the button class back to its default state
        } else {
          button.className = 'btn-err';
          const message = await response.text();
          button.textContent = 'failed with status ' + response.status + ' ' + message;
        }
      } catch (e) {
        button.textcontent = 'failed with error ' + e;
        console.error(e);
      }
    }
  });
</script>

[Dieterbe]

are you saying you want to block rendering of any code?
i think ability to show code blocks is useful.
e.g. check out some of the comments here: https://dieter.plaetinck.be/posts/poor_mans_dmenu_benchmark/

[emeralit]

Not, indeed. We don't want save malicious code. This is an example of how the code has been saved

2024-04-09-il-gioco-della-nostalgia2024-04-15T16:56:06.088588+02:00### Code <code>&lt;img src="https://cdn.icon-icons.com/icons2/1159/PNG/128/apple-rainbow_81608.png&#34; onload="console.log('unsafe code injected!')";&gt;</code>Mimmodomipri@gmail.com

How it appears
<img src="https://cdn.icon-icons.com/icons2/1159/PNG/128/apple-rainbow_81608.png" onload="console.log('unsafe code injected!')";>