Ben Chuanlong Du's Blog

It is never too late to learn.

Parse TOML Files in Golang

Things on this page are fragmentary and immature notes/thoughts of the author. Please read with your own judgement!

Tips and Traps

  1. If the schema of a TOML file is unknown, you can parse it into a generic object (any or map[string]any). When parsing a TOML file into a generic object, map is used for storing key-value pairs. Since map is unordered, the order of keys in the original TOML file is not preserved. Using a ordered map solves the problem partly. For example, parsing a TOML file into a orderedmap.OrderedMap[string, any] preserved the order of keys in the outer-most map but not in inner maps.
In [35]:
import "io/ioutil"
import "fmt"
import "reflect"
import "github.com/pelletier/go-toml/v2"
In [2]:
blob := []byte(`
key1 = "value1"
key2 = "value2"
key3 = "value3"
`)
var dat map[string]interface{}
toml.Unmarshal(blob, &dat)
dat
Out[2]:
map[key1:value1 key2:value2 key3:value3]
In [3]:
bytes, err := ioutil.ReadFile("pyproject.toml")
bytes
Out[3]:
[91 116 111 111 108 46 112 111 101 116 114 121 93 10 110 97 109 101 32 61 32 34 97 114 114 111 119 100 98 34 10 118 101 114 115 105 111 110 32 61 32 34 48 46 49 46 48 34 10 100 101 115 99 114 105 112 116 105 111 110 32 61 32 34 81 117 101 114 121 105 110 103 32 100 97 116 97 98 97 115 101 32 117 115 105 110 103 32 97 114 114 111 119 46 106 118 109 46 34 10 97 117 116 104 111 114 115 32 61 32 91 34 66 101 110 32 68 117 32 60 108 111 110 103 101 110 100 117 64 121 97 104 111 111 46 99 111 109 62 34 93 10 10 91 116 111 111 108 46 112 111 101 116 114 121 46 100 101 112 101 110 100 101 110 99 105 101 115 93 10 112 121 116 104 111 110 32 61 32 34 62 61 51 46 56 44 60 52 46 48 34 10 110 117 109 112 121 32 61 32 34 62 61 49 46 50 50 46 52 34 10 112 97 110 100 97 115 32 61 32 34 62 61 48 46 50 50 34 10 74 80 121 112 101 49 32 61 32 34 62 61 49 46 49 46 50 34 10 115 113 108 112 97 114 115 101 32 61 32 34 62 61 48 46 50 46 52 34 10 108 111 103 117 114 117 32 61 32 34 62 61 48 46 50 46 53 34 10 112 97 110 100 97 115 45 112 114 111 102 105 108 105 110 103 32 61 32 34 62 61 50 46 56 46 48 34 10 112 121 97 114 114 111 119 32 61 32 34 62 61 50 46 48 46 48 34 10 10 91 116 111 111 108 46 112 111 101 116 114 121 46 100 101 118 45 100 101 112 101 110 100 101 110 99 105 101 115 93 10 112 121 116 101 115 116 32 61 32 34 62 61 51 46 48 34 10 112 121 108 105 110 116 32 61 32 34 62 61 50 46 49 50 46 50 34 10 112 121 116 121 112 101 32 61 32 34 62 61 50 48 50 48 46 48 46 48 34 10 121 97 112 102 32 61 32 34 62 61 48 46 51 50 46 48 34 10 116 111 109 108 32 61 32 34 62 61 48 46 49 48 46 50 34 10 10 91 116 111 111 108 46 121 97 112 102 93 10 98 97 115 101 100 95 111 110 95 115 116 121 108 101 32 61 32 34 102 97 99 101 98 111 111 107 34 10 99 111 108 117 109 110 95 108 105 109 105 116 32 61 32 56 56 10 10 91 116 111 111 108 46 121 97 112 102 105 103 110 111 114 101 93 10 105 103 110 111 114 101 95 112 97 116 116 101 114 110 115 32 61 32 91 10 32 32 32 32 34 42 47 46 105 112 121 110 98 95 99 104 101 99 107 112 111 105 110 116 115 47 34 44 10 32 32 32 32 34 46 118 101 110 118 47 34 44 10 32 32 32 32 34 118 101 110 118 47 34 44 10 93 10 10 91 116 111 111 108 46 112 121 114 105 103 104 116 93 10 105 110 99 108 117 100 101 32 61 32 91 34 97 114 114 111 119 100 98 34 44 32 34 116 101 115 116 115 34 93 10 101 120 99 108 117 100 101 32 61 32 91 10 32 32 32 32 34 42 42 47 110 111 100 101 95 109 111 100 117 108 101 115 34 44 10 32 32 32 32 34 42 42 47 95 95 112 121 99 97 99 104 101 95 95 34 44 10 32 32 32 32 34 42 42 47 116 121 112 101 115 116 117 98 115 34 44 10 32 32 32 32 34 42 42 47 46 112 121 116 121 112 101 34 44 10 32 32 32 32 34 46 118 101 110 118 34 44 10 93 10 118 101 110 118 80 97 116 104 32 61 32 34 46 34 10 118 101 110 118 32 61 32 34 46 118 101 110 118 34 10 10 114 101 112 111 114 116 77 105 115 115 105 110 103 73 109 112 111 114 116 115 32 61 32 116 114 117 101 10 114 101 112 111 114 116 77 105 115 115 105 110 103 84 121 112 101 83 116 117 98 115 32 61 32 102 97 108 115 101 10 10 91 98 117 105 108 100 45 115 121 115 116 101 109 93 10 114 101 113 117 105 114 101 115 32 61 32 91 34 112 111 101 116 114 121 62 61 48 46 49 50 34 93 10 98 117 105 108 100 45 98 97 99 107 101 110 100 32 61 32 34 112 111 101 116 114 121 46 109 97 115 111 110 114 121 46 97 112 105 34 10]
In [4]:
var conf map[string]interface{}
toml.Unmarshal(bytes, &conf)
conf
Out[4]:
map[build-system:map[build-backend:poetry.masonry.api requires:[poetry>=0.12]] tool:map[poetry:map[authors:[Ben Du <longendu@yahoo.com>] dependencies:map[JPype1:>=1.1.2 loguru:>=0.2.5 numpy:>=1.22.4 pandas:>=0.22 pandas-profiling:>=2.8.0 pyarrow:>=2.0.0 python:>=3.8,<4.0 sqlparse:>=0.2.4] description:Querying database using arrow.jvm. dev-dependencies:map[pylint:>=2.12.2 pytest:>=3.0 pytype:>=2020.0.0 toml:>=0.10.2 yapf:>=0.32.0] name:arrowdb version:0.1.0] pyright:map[exclude:[**/node_modules **/__pycache__ **/typestubs **/.pytype .venv] include:[arrowdb tests] reportMissingImports:true reportMissingTypeStubs:false venv:.venv venvPath:.] yapf:map[based_on_style:facebook column_limit:88] yapfignore:map[ignore_patterns:[*/.ipynb_checkpoints/ .venv/ venv/]]]]
In [5]:
reflect.TypeOf(conf)
Out[5]:
map[string]interface {}
In [12]:
b, err := conf["build-system"]
b
Out[12]:
map[build-backend:poetry.masonry.api requires:[poetry>=0.12]]
In [18]:
_, isMap := b.(map[string]interface{})
In [19]:
isMap
Out[19]:
true
In [21]:
_, isArr := b.([]interface{})
In [22]:
isArr
Out[22]:
false
In [32]:
b
Out[32]:
map[build-backend:poetry.masonry.api requires:[poetry>=0.12]]
In [33]:
b.(map[string]interface{})["build-backend"]
Out[33]:
poetry.masonry.apitrue
In [37]:
switch v := b.(type) {
case map[string]interface{}:
    conf["build-system"] = v
case []interface{}:
    fmt.Printf("array\n")
default:
    fmt.Printf("other types")
}
In [42]:
c, err := conf["build-system"]
c
Out[42]:
map[build-backend:poetry.masonry.api requires:[poetry>=0.12]]
In [49]:
c["build-backend"]

repl.go:1:3: invalid operation: c["build-backend"] (type interface{} does not support indexing)
In [43]:
reflect.TypeOf(c)
Out[43]:
map[string]interface {}
In [46]:
t, err := conf["tool"]
t
Out[46]:
map[poetry:map[authors:[Ben Du <longendu@yahoo.com>] dependencies:map[JPype1:>=1.1.2 loguru:>=0.2.5 numpy:>=1.22.4 pandas:>=0.22 pandas-profiling:>=2.8.0 pyarrow:>=2.0.0 python:>=3.8,<4.0 sqlparse:>=0.2.4] description:Querying database using arrow.jvm. dev-dependencies:map[pylint:>=2.12.2 pytest:>=3.0 pytype:>=2020.0.0 toml:>=0.10.2 yapf:>=0.32.0] name:arrowdb version:0.1.0] pyright:map[exclude:[**/node_modules **/__pycache__ **/typestubs **/.pytype .venv] include:[arrowdb tests] reportMissingImports:true reportMissingTypeStubs:false venv:.venv venvPath:.] yapf:map[based_on_style:facebook column_limit:88] yapfignore:map[ignore_patterns:[*/.ipynb_checkpoints/ .venv/ venv/]]]
In [47]:
reflect.TypeOf(t)
Out[47]:
map[string]interface {}
In [48]:
t["poetry"]

repl.go:1:3: invalid operation: t["poetry"] (type interface{} does not support indexing)
In [ ]:
bm := map[string]interface{}(b)
In [7]:
reflect.TypeOf(conf["build-system"])

repl.go:1:21: not enough arguments in call to reflect.TypeOf:
	have (interface{})
	want (interface{})
In [10]:
bytes, err := toml.Marshal(conf)
string(bytes)
Out[10]:
[build-system]
build-backend = 'poetry.masonry.api'
requires = ['poetry>=0.12']

[tool]
[tool.poetry]
authors = ['Ben Du <longendu@yahoo.com>']
description = 'Querying database using arrow.jvm.'
name = 'arrowdb'
version = '0.1.0'
[tool.poetry.dependencies]
JPype1 = '>=1.1.2'
loguru = '>=0.2.5'
numpy = '>=1.22.4'
pandas = '>=0.22'
pandas-profiling = '>=2.8.0'
pyarrow = '>=2.0.0'
python = '>=3.8,<4.0'
sqlparse = '>=0.2.4'

[tool.poetry.dev-dependencies]
pylint = '>=2.12.2'
pytest = '>=3.0'
pytype = '>=2020.0.0'
toml = '>=0.10.2'
yapf = '>=0.32.0'


[tool.pyright]
exclude = ['**/node_modules', '**/__pycache__', '**/typestubs', '**/.pytype', '.venv']
include = ['arrowdb', 'tests']
reportMissingImports = true
reportMissingTypeStubs = false
venv = '.venv'
venvPath = '.'

[tool.yapf]
based_on_style = 'facebook'
column_limit = 88

[tool.yapfignore]
ignore_patterns = ['*/.ipynb_checkpoints/', '.venv/', 'venv/']

References

Official doc - go-toml

In [ ]:

Comments