Where to store remote data in Svelte for responsive data vizualisations with Svelte and D3 (best pra-CodePudding

Disclaimer: I am really not well experienced with neither svelte, nor D3, nor general JavaScript patterns. But I really like it, so I really want to learn it and already invested quite some time. Still, this feels like a super basic question that annoys me a lot. I hope it is not too confusing and someone might has an idea.

It is basically about how to setup a simple graph (let it be a bar chart) in an efficient, reproducible and "best-practice" way. I guess my main concern is on how to pass around the data and use it for different tasks. E.g. I think it might be a good idea to separate out the construction of the scales (using d3) in a separate component. However, this component needs access to the data (and probably also access to the, in the best case resposive width, of the chart-container).

However, also the bars, which are in another component, need access to the data in order to know how do draw the rectangles.

A general misunderstanding (i guess that is the right word) I have with JavaScript is that I do not understand how to fetch data asynchronously (using e.g. the browsers fetchor D3's csvmethod). I simply can not fetch the data and then pass it as prop to another component. Because what I would be passing would be a promise...

So I have this very basic REPL that kind of shows a bit this know I have in my head: https://svelte.dev/repl/398f4c21b7a9409a9811fd8e38703a36?version=3.44.1

It looks like this. In the App.html I fetch the data that I want to use for multiple purposes. However I cannot "get it out" of that component.

<script>
    import Chart from "./Chart.svelte"

    
    const url = "https://api.github.com/search/repositories?q=stars:>100000";
    
    async function getData(){
        let response = await fetch(url)
        let data = await response.json()
        console.log(data)
    }
    
    //async function getDataLocal(){
    //  let data = await d3.csv(<path_to_data>)
    //      return await data
    //  }
    
    
    let data = await getData()
    
</script>

<Chart {data}>Do Something with the data. Make the chart, build the scales, ....</Chart>

So the main questions are:

Are there any ressources on how to learn building sustainable graphics with remote data, svelte and a bit of D3. I already watched many many youtube videos and I guess I will rewatch the one from Matthias Stahl;)
Is it a good idea to use stores in such a case to store the data
And a little more specific: As the data is (probably) fixed, however the dimension arent't: What is a good way/place to let the app know to recalculate the scales etc.

CodePudding user response：

There are 3 separate concerns here:

fetching, storing and retrieving data (aka the data source layer)
manipulating/transforming data (aka the business logic layer)
displaying data (aka the presentation layer)

I will leave the last part aside as it solely concerns D3 (if that is your visualization library of choice) and there are plenty of resources available online on this topic, and I will instead focus on what seems to be the heart of your question, i.e. how to fetch data in Svelte, where to store it, how to pass it around to components, and how to manipulate the data.

1. Asynchronous queries in Svelte

Your first inquiry is about how to deal with asynchronous requests. You cannot use await statements at the root level of the <script> section of a Svelte file, meaning the following reactive statement would generate an error:

// will fail
$: data = await getData(url)

However, you can call an asynchronous function that will handle the assignment. Reactivity will still work and your component will re-render when the url is changed and the new data retrieved:

// will work
$: updateData(url)

async function updateData(url) {
  data = await getData(url)
}

Here is a working example based on the REPL in your question

2. Using stores

As you could see from the above example, you had to pass the data to your <Header> and <Chart> components for it to be used in either:

<Header {data}>GitHub Lookup</Header>
<Chart {data}/>

But what if you want to use your Chart somewhere else in your application? What if you have another component that wants to make use of the same data?

Obviously you do not want to fetch the same data over & over (unless the request itself has changed). You also want to avoid passing the data around as a prop everywhere in your app. You will want to make the data available only to these components that will use it.

This is where stores come in handy. Stores can be subscribed to by any component. A writable store will allow its contents to be updated, while a readable store will be -as the name implies- read-only.

A store need not be complex. The following is a very basic writable store:

import { writable } from 'svelte/store'

export const githubStore = writable(null) // initialized with a null value

All you have to do then is interact with your store.

Updating the store in your App component:

import { githubStore as data } from './githubStore.js' // import the store we defined above under the name 'data'
.
.
.
async function updateData(url) {
  $data = await getData(url) // using the $ shorthand to access the store (assigning a new value will update the store content)
}

Using (i.e. subscribing to) the store in your components:

import { githubStore as data } from './githubStore.js' // import the store we defined above under the name 'data'
.
.
.
// using the $ shorthand to access the store
{#each $data.items as item (item.id)}
  <li><a href={item.html_url}>{item.full_name}</a> [{item.stargazers_count}⭐]</li>
{/each}

Read here for details on using the $ reactive syntax with stores

Now that your child components are subscribing to the store where you stored your data, you do not need to pass that data as a prop any more:

<Header>GitHub Lookup</Header>
<Chart />

Here is an updated version of the REPL above, using stores

3. Further considerations

When you want to start manipulating or transforming data that has been put into a store, derived stores come in handy. When the data in your original store is updated, the derived store will automatically update itself based on the changes to the original.

You can also build on the provided readable/writable stores by adding your own functionality and custom methods. These are slightly more advanced topics but would come in handy where data manipulation is concerned.

Finally, D3 will provide its own data manipulation methods, so it will be up to you to decide how much manipulation you handle directly in Svelte, and how much you delegate to D3. I would probably leave everything connected to visualization (scaling, zooming, etc.) on the D3 side, and have the actual pre-visualization manipulation of data (i.e. the business logic) on the Svelte side (or better yet, directly on the back-end if you have access to that!).