Golang reads html tags (<>) from JSON string data as &lt and &gt which causes rendering issues-CodePudding

I have a basic web server that renders blog posts from a database of JSON posts wherein the main paragraphs are built from a JSON string array. I was trying to find a way to easily encode new lines or line breaks and found a lot of difficulty with how the encoding for these values changes from JSON to GoLang and finally to my HTML webpage. When I tried to encode my JSON with newlines I found I had to encode them using \\n rather than just \n in order for them to actually appear on my page. One problem however was they simply appeared as text and not line breaks.

I then tried to research ways to replace the \n portions of the joined string array into   tags, however I could not find any way to do this with go and moved to trying to do so in javascript. This did not work either despite me deferring the calling of my javascript in my link from my HTML. this is that javascript:

var title = window.document.getElementById("title");
var timestamp = window.document.getElementById("timestamp");
var sitemap = window.document.getElementById("sitemap");
var main = window.document.getElementById("main");
var contact_form = window.document.getElementById("contact-form");
var content_info = window.document.getElementById("content-info");

var str = main.innerHTML;

function replaceNewlines() {
    // Replace the \n with <br>
    str = str.replace(/(?:\r\n|\r|\n)/g, "<br>");

    // Update the value of paragraph
    main.innerHTML = str;
}

Here is my HTML:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Dynamic JSON Events</title>
    <link rel="stylesheet" href="/blogtemplate.css"></style>
</head>
<body>
    <section id="title">
        <h1 id="text-title">{{.Title}}</h1>
        <time id="timestamp">
            {{.Timestamp}}
        </time>
    </section>
    <nav role="navigation" id="site-nav">
        <ul id="sitemap">
        </ul>
    </nav>
    <main role="main" id="main">
        {{.ParsedMain}}
    </main>
    <footer role="contentinfo" id="footer">
        <form id="contact-form" role="form">
        <address>
            Contact me by <a id="my-email" href="mailto:[email protected]" >e-mail</a>
        </address>
        </form>
    </footer>
<script defer src="/blogtemplate.js">
</script>
</body>
</html>

I then finally turned to trying to hardcode   tags into my json data to discover that this simply renders as &lt and &gt when it finally reaches the browser. I am getting pretty frustrated with this process of encoding constantly causing me issues in creating newlines and line breaks. How can I easily include newlines where I want in my JSON string data?

Here is my Go script if it helps:

package main

import (
    "encoding/json"
    "html/template"
    "log"
    "net/http"
    "os"
    "regexp"
    "strings"
)

type BlogPost struct {
    Title      string   `json:"title"`
    Timestamp  string   `json:"timestamp"`
    Main       []string `json:"main"`
    ParsedMain string
}

// this did not seem to work when I tried to implement it below
var re = regexp.MustCompile(`\r\n|[\r\n\v\f\x{0085}\x{2028}\x{2029}]`)
func replaceRegexp(s string) string {
    return re.ReplaceAllString(s, "<br>\n")
}

var blogTemplate = template.Must(template.ParseFiles("./assets/docs/blogtemplate.html"))

func blogHandler(w http.ResponseWriter, r *http.Request) {
    blogstr := r.URL.Path[len("/blog/"):]   ".json"

    f, err := os.Open("db/"   blogstr)
    if err != nil {
        http.Error(w, err.Error(), http.StatusNotFound)
        return
    }
    defer f.Close()

    var post BlogPost
    if err := json.NewDecoder(f).Decode(&post); err != nil {
        http.Error(w, err.Error(), http.StatusInternalServerError)
        return
    }

    post.ParsedMain = strings.Join(post.Main, "")

    // post.ParsedMain = replaceRegexp(post.ParsedMain)

    if err := blogTemplate.Execute(w, post); err != nil {
        log.Println(err)
    }
}

func teapotHandler(w http.ResponseWriter, r *http.Request) {
    w.WriteHeader(http.StatusTeapot)
    w.Write([]byte("<html><h1><a href='https://datatracker.ietf.org/doc/html/rfc2324/'>HTCPTP</h1><img src='https://external-content.duckduckgo.com/iu/?u=https://taooftea.com/wp-content/uploads/2015/12/yixing-dark-brown-small.jpg&f=1&nofb=1' alt='Im a teapot'><html>"))
}

func faviconHandler(w http.ResponseWriter, r *http.Request) {
    http.ServeFile(w, r, "./assets/art/favicon.ico")
}

func main() {
    http.Handle("/", http.FileServer(http.Dir("/assets/docs")))
    http.HandleFunc("/blog/", blogHandler)
    http.HandleFunc("/favicon.ico", faviconHandler)
    http.HandleFunc("/teapot", teapotHandler)
    log.Fatal(http.ListenAndServe(":8080", nil))
}

Here is an example of my JSON data:

{
    "title" : "Finished My First Blog App",
    "timestamp": "Friday, March 18th, 11:39 AM",
    "main": [
        "It took me awhile to tidy everything up but I finally finished creating my first ",
        "blog app using Go along with JSON for my database. I plan on using this to document ",
        "my own thoughts and experiences as a programmer and cybersecurity researcher; things ",
        "like tutorials, thought-pieces, and journals on my own projects progress will be ",
        "posted here. I look forward to getting more used to writing and sharing my own story, ",
        "I think it will help me learn from doing and also hearing feedback from others.\\n\\n",
        "I utilized a handler function to dynamically read from my JSON database and template ",
        "data into my HTML template using the go html/template package as well as the encoding/json ",
        "to handling reading those objects. Next I had to make sure my CSS and JavaScript assets ",
        "would be served alongside this finished template in order for my styling to be output into ",
        "the browser. For this I used a FileServer function which allowed for me to serve linked ",
        "resources in my HTML boilerplate and have the server still locate blog resources dynamically. ",
        "Going forward I am looking to add better styling, more JavaScript elements to the page, and ",
        "more functionality to how my JSON data is encoded and parsed in order to create more complex ",
        "looking pages and blog posts."
    ]
}

I am just trying to find a way to easily include spaces between paragraphs in the long array of strings in my JSON however I have failed in Go, my JS doesn't ever seem to affect my webpage(this is not the only problem I have had with this, it does not seem to want to affect any page elements for some reason), and I cannot seem to hardcode   tags directly into my JSON as the browser interprets those as  . Nothing I have tried has actually let me encode linebreaks, What can I do here?

CodePudding user response：

You could try to loop over your array inside the template and generate a p tag for every element of the array. This way there is no need to edit your main array in go.

Template:

<!DOCTYPE html>
<html lang="en">
<head>
    <meta charset="UTF-8">
    <meta http-equiv="X-UA-Compatible" content="IE=edge">
    <meta name="viewport" content="width=device-width, initial-scale=1.0">
    <title>Dynamic JSON Events</title>
    <link rel="stylesheet" href="/blogtemplate.css"></style>
</head>
<body>
    <section id="title">
        <h1 id="text-title">{{.Title}}</h1>
        <time id="timestamp">
            {{.Timestamp}}
        </time>
    </section>
    <nav role="navigation" id="site-nav">
        <ul id="sitemap">
        </ul>
    </nav>
    <main role="main" id="main">
        {{range $element := .Main}} <p>{{$element}}</p> {{end}}
    </main>
    <footer role="contentinfo" id="footer">
        <form id="contact-form" role="form">
        <address>
            Contact me by <a id="my-email" href="mailto:[email protected]" >e-mail</a>
        </address>
        </form>
    </footer>
<script defer src="/blogtemplate.js">
</script>
</body>
</html>

CodePudding user response：

If you want the JSON document to contain HTML, then do the following:

Change ParsedMain to type html.HTML:

 type BlogPost struct {
     ...
     ParsedMain html.HTML
 }

Convert to that type when assigning the field:

 post.ParsedMain = html.HTML(strings.Join(post.Main, "")).

Replace the backslash and n sequences in the document with  .

If untrusted users can enter the JSON document data, then the application should filter the HTML tags and attributes against an allow list. This prevents an attacker mounting an attack through script injection.

If you want translate newlines in the JSON document to line breaks in HTML, then do the following:

Change the document to actually include newlines: \\n -> \n.

On the server or the client, replace the newlines with an HTML line break. Here's how to do it on the server:

 type BlogPost struct {
     ...
     ParsedMain html.HTML
 }

 post.ParsedMain = html.HTML(strings.ReplaceAll(strings.Join(post.Main, ""), "\n", "<br>"))).

It looks like you already know how to translate newlines to line breaks on the client.

You may want to use  instead of  .