JSON (JavaScript Object Notation) is a human readable data and file format commonly used to exchange data, in particular for web applications. As such, it is no surprise that is holds a special place in the Elm language.
Elm provides the core modules Json.Decode
and Json.Encode
to parse and write JSON data.
JSON Decoders are a way to declare what type of data is expected and which data structure the data should map to.
Let's import some aliases and types from the modules:
import Json.Decode as Decode exposing (Decoder, Error)
import Json.Encode as Encode exposing (Value)
Using Decode.decodeString : Decoder a -> String -> Result Error a
with a decoder and a JSON string will either succeed if the string confirms to the decoder specifications, or fail otherwise.
There is no way of inspecting the raw JSON data when defining the decoder, which could feel unusual coming from imperative languages.
JSON has a small number of core data types:
"hello world"
true
or false
34
or 0.12
null
Each of these is associated with a Decode
function:
Decode.decodeString Decode.string """\"hello\""""
--> Ok "hello"
Decode.decodeString Decode.string """null"""
--> Err ...
Decode.decodeString Decode.bool """true"""
--> Ok True
Decode.decodeString Decode.bool """0"""
--> Err ...
Decode.decodeString Decode.int """12"""
--> Ok 12
Decode.decodeString Decode.float """3.14"""
--> Ok 3.14
Decode.decodeString (Decode.null AnyValue) """null"""
--> Ok AnyValue
Decode.decodeString (Decode.null AnyValue) """true"""
--> Err ...
Note that Decode.null
lets you decide how to model a null
in your program by requiring an arbitrary value as argument (maybe ()
, Nothing
, or anything appropriate to your program).
JSON also defines two data structures to collect core data types:
[0, null, "none", []]
{"id": 1, "is_admin": false, "more_info": {}}
Decode
also offers functions to decode these:
Decode.decodeString (Decode.list Decode.int) """[1, 2, 3]"""
--> Ok [1, 2, 3]
Decode.decodeString (Decode.list Decode.int) """[1, 2, 3, "not an int"]"""
--> Err ...
Decode.decodeString (Decode.dict Decode.int) """{ "key1": 17, "key2": 71 }"""
--> Ok (Dict.fromList [("key1", 17), ("key2", 71)])
Decode.decodeString (Decode.dict Decode.int) """{ "key1": 17, "key2": "seventy-one" }"""
--> Err ...
These functions Decode.list : Decoder a -> Decoder (List a)
and Decode.dict : Decoder a -> Decoder (Dict String a)
each expect a decoder as argument, which is expected to decode every single element elements of the data structure.
If the argument decoder fails to decode any element of an array or object value, the whole decoder fails, which is a common pattern.
Combining simple decoders into more complex ones is the core idea of the technique. We will introduce several more functions that can be used to build arbitrarily complex real-world data decoders through means of an example.
For practicing combining basic decoders to parse complex JSON data, let's build decoders for parsing (a subset of) GeoJSON data. GeoJSON is a specialized JSON-based data format used to represent geographic features, such as locations, paths or regions, specified with 2D or 3D geographic coordinates, along with other properties.
A GeoJSON Geometry Object is an object that contains a type
field with a string value (there are 7 possible type values) and a coordinates
field with geographic coordinates (collected in a structure that depends on the type).
For example,
{
"type": "Point",
"coordinates": [127.831, 26.461]
}
or
{
"type": "LineString",
"coordinates": [
[127.831, 26.461],
[127.829, 26.465],
[127.829, 26.469],
[127.83, 26.47]
]
}
A Geometry Object could be represented in Elm with the following type
type Geometry
= Point (List Float)
| LineString (List (List Float))
| ...
To probe the value of a certain field, we can use Decode.field : String -> Decoder a -> Decoder a
which decodes the value of a specific key of an object.
decodePointCoordinates : Decoder (List Float)
decodePointCoordinates =
Decode.field "coordinates" (Decode.list Decode.float)
decodeLineStringCoordinates : Decoder (List (List Float))
decodeLineStringCoordinates =
Decode.field "coordinates" (Decode.list (Decode.list Decode.float))
To transform the value successfully parsed by a decoder, we can use Decode.map : (a -> value) -> Decoder a -> Decoder value
, or any of its siblings, from Decode.map2 (a -> b -> value) -> Decoder a -> Decoder b -> Decoder value
to Decode.map8
.
In this case, we want to decide which coordinate decoder to use depending on the value in the type
field, which is a perfect job for Decode.andThen : (a -> Decoder b) -> Decoder a -> Decoder b
and its friends Decode.succeed : a -> Decoder a
and Decode.fail : String -> Decoder a
.
decodeGeometry : Decoder Geometry
decodeGeometry =
Decode.field "type" Decode.string
|> Decode.andThen
(\value ->
case value of
"Point" ->
decodePointCoordinates |> Decode.map Point
"LineString" ->
decodeLineStringCoordinates |> Decode.map LineString
_ ->
Decode.fail "Geometry not implemented yet, or invalid type value"
)
Note at this point that we could use Decode.andThen
to validate the coordinates (checking that the coordinates come in pairs or triplets and checking that the coordinate values make geographic sense).
Decode.decodeString decodeGeometry """{"type": "Point", "coordinates": [127.831, 26.461]}"""
--> Ok (Point [127.831, 26.461])
Decode.decodeString decodeGeometry """{"type": "LineString", "coordinates": [127.831, 26.461]}"""
--> Err ...
A GeoJSON Feature Object represents something spatially bounded: it has a type
field with the "Feature"
value, a geometry
field with a Geometry Object or a null
, a properties
field with an arbitrary JSON Object or a null
, and an optional id
field with a JSON string or number value.
{
"type": "Feature",
"geometry": null,
"properties": null
}
or
{
"type": "Feature",
"id": "0157",
"geometry": {
"type": "Point",
"coordinates": [127.831, 26.461]
},
"properties": {
"country": "Japan"
}
}
Let's tackle the id
field first.
If it exists, it must accommodate strings or numbers, so we could either introduce a custom type for the value, or always save is as a string.
To choose between two possible decoders, we can use Decode.oneOf : List (Decoder a) -> Decoder a
.
decodeFeatureId : Decoder String
decodeFeatureId =
Decode.oneOf
[ Decode.string
, Decode.int |> Decode.map String.fromInt
, Decode.float |> Decode.map String.fromFloat
]
|> Decode.field "id"
Decode.decodeString decodeFeatureId """{"id": "seventeen"}"""
--> Ok "seventeen"
Decode.decodeString decodeFeatureId """{"id": 17}"""
--> Ok "17"
Decode.decodeString decodeFeatureId """{"type": "Point"}"""
--> Er ...
To express that the field is optional, we could use the same technique
decodeMaybeFeatureId : Decoder (Maybe String)
decodeMaybeFeatureId =
Decode.oneOf
[ decodeFeatureId |> Decode.map Just
, Decode.succeed Nothing
]
But we could instead use Decode.maybe : Decoder a -> Decoder (Maybe a)
decodeMaybeFeatureId : Decoder (Maybe String)
decodeMaybeFeatureId =
Decode.maybe decodeFeatureId
Decode.decodeString decodeMaybeFeatureId """{"id": 17}"""
--> Ok (Just "17")
Decode.decodeString decodeMaybeFeatureId """{"type": "Point"}"""
--> Ok Nothing
Note that decodeMaybeFeatureId
(or any decoder wrapped with Decode.maybe
) will always succeed since it can always fallback on Ok Nothing
, although a decoder that uses it might still fail
Decode.decodeString (Decode.list decodeMaybeFeatureId) """{"id": 17}"""
--> Err ...
Next, let's focus on the geometry
field, which can either be a Geometry Object or be null
.
We could use Decode.oneOf
once again, but there is a better option: Decode.nullable : Decoder a -> Decoder (Maybe a)
.
decodeFeatureGeometry : Decoder (Maybe Geometry)
decodeFeatureGeometry =
Decode.nullable decodeGeometry
|> Decode.field "geometry"
Decode.decodeString decodeFeatureGeometry """{"geometry": {"type": "Point", "coordinates": [127.831, 26.461]}}"""
--> Ok (Just (Point [127.831, 26.461]))
Decode.decodeString decodeFeatureGeometry """{"geometry": null}"""
--> Ok Nothing
Decode.decodeString decodeFeatureGeometry """{"geometry": {}}"""
--> Err ...
Decode.maybe
and Decode.nullable
have the same signature, but their behavior is different, since the OK Nothing
result of a Decode.nullable
will only come from the specific JSON value null
.
Finally, the properties
field can contain an arbitrary JSON Object or a null
, which is a job for Decode.value : Decoder Value
.
decodeProperties : Decoder (Maybe (Dict String Value))
decodeProperties =
Decode.nullable (Decode.dict Decode.value)
|> Decode.field "properties"
Decode.decodeString decodeProperties """{"properties": null}"""
--> Ok Nothing
Decode.decodeString decodeProperties """{"properties": {"country": "Japan"}}"""
--> Ok (Just (Dict.fromList [("country", <internals>)]))
Decode.decodeString decodeProperties """{"properties": 17}"""
--> Err ...
Note that Value
is an opaque type, which means you cannot easily probe its content.
To use it, you can either write a decoder for it and run it with Decode.decodeValue : Decoder a -> Value -> Result Error a
, or send it to JavaScript directly via a port.
We are ready to parse Feature Objects:
type alias Feature =
{ id : Maybe String
, geometry : Maybe Geometry
, properties : Maybe (Dict String Value)
}
decodeFeature : Decoder Feature
decodeFeature =
Decode.field "type" Decode.string
|> Decode.andThen
(\value ->
if value /= "Feature" then
Decode.fail "not a Feature"
else
Decode.map3 Feature
decodeMaybeFeatureId
decodeFeatureGeometry
decodeProperties
)
Decode.decodeString decodeFeature """{"type": "Feature", "geometry": null, "properties": null}"""
--> Ok { geometry = Nothing, id = Nothing, properties = Nothing }
"""
{
"type": "Feature",
"id": "0157",
"geometry": {
"type": "Point",
"coordinates": [127.831, 26.461]
},
"properties": {
"country": "Japan"
}
}
"""
|> Decode.decodeString decodeFeature
--> Ok { geometry = Just (Point [127.831,26.461]), id = Just "0157", properties = Just (Dict.fromList [("country",<internals>)]) }
Encoders allow to write valid JSON from Elm values using the Encode.encode : Int -> Value -> String
function.
The first argument specifies the amount of indentation in the final result, and the second argument is the JSON value to write.
A Value
can either be obtained from Decode.value
or be produced from one of the encoders:
Encode.encode 0 (Encode.string "hello")
--> "hello"
Encode.encode 0 (Encode.bool True)
--> "true"
Encode.encode 0 (Encode.int 12)
--> "12"
Encode.encode 0 (Encode.float 3.14)
--> "3.14"
Encode.encode 0 Encode.null
--> "null"
Encode.encode 0 (Encode.list Encode.int [1, 2, 3])
--> "[1,2,3]"
Encode.encode 4 (Encode.list Encode.int [1, 2, 3])
--> "[\n 1,\n 2,\n 3\n]"
Encode.encode 0 (Encode.dict String.toLower Encode.int (Dict.fromList [("KEY1", 17), ("KEY2", 71)]))
--> "{\"key1\":17,\"key2\":71}"
Encode.encode 4 (Encode.dict String.toLower Encode.int (Dict.fromList [("KEY1", 17), ("KEY2", 71)]))
--> "{\n \"key1\": 17,\n \"key2\": 71\n}"
as well as with Encode.object : List ( String, Value ) -> Value
Encode.object
[ ( "key1", Encode.int 17 )
, ( "key2", Encode.int 71 )
]
|> Encode.encode 0
--> "{\"key1\":17,\"key2\":71}"
Le's define encoders for the GeoJSON decoders defined earlier
encodeGeometry : Geometry -> Value
encodeGeometry geometry =
case geometry of
Point coord ->
Encode.object
[ ( "type", Encode.string "Point" )
, ( "coordinates", Encode.list Encode.float coord )
]
LineString coord ->
Encode.object
[ ( "type", Encode.string "LineString" )
, ( "coordinates", Encode.list (Encode.list Encode.float) coord )
]
encodeFeature : Feature -> Value
encodeFeature feature =
let
maybeId =
case feature.id of
Nothing ->
[]
Just id ->
[ ( "id", Encode.string id ) ]
in
Encode.object
(maybeId
++ [ ( "type", Encode.string "Feature" )
, ( "geometry"
, case feature.geometry of
Nothing ->
Encode.null
Just geometry ->
encodeGeometry geometry
)
, ( "properties"
, case feature.properties of
Nothing ->
Encode.null
Just dict ->
Encode.dict identity identity dict
)
]
)
In general, a good test for your decoder-encoder pairs is checking that elmValue |> encoder |> decoder == elmValue
.
The more you learn about programming, the more you start thinking that this might be more than just a fad. There might even be some business opportunities: people write code, they want to save it, share it, collaborate, maybe get some feedback even.
Alright, let's do it, let's build a service to encourage people to code: hup, hup, people, start using git! Let's call it... GitHup.
You get started on building a REST JSON API: users, pull requests, comments, the backend is swiftly done. Here is a example of the payload being served when you ask the server for the list of comments on a pull request:
[
{
"id": 256,
"pull_request_review_id": 42,
"user": {
"id": 101,
"login": "octodog",
"avatar_url": "https://githup.com/images/error/octodog_happy.gif",
"site_admin": false
},
"body": "Great stuff!",
"side": "RIGHT",
"_links": {
"self": {
"href": "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1"
},
"html": {
"href": "https://githup.com/octodog/Hello-World/pull/1#discussion-diff-1"
},
"pull_request": {
"href": "https://api.githup.com/repos/octodog/Hello-World/pulls/1"
}
}
},
{
"id": 11,
"pull_request_review_id": null,
"user": {
"id": 2,
"login": "hexacat",
"name": "Alex Kate",
"avatar_url": "https://githup.com/images/error/hexacat_happy.gif",
"site_admin": true
},
"body": "Amazing stuff!",
"side": "LEFT",
"_links": {
"self": {
"href": "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1"
}
}
}
]
Of course, the frontend will be done in Elm, but first you need to figure out what those notorious decoders/encoders are all about.
Humorous story aside, the exercise requirements are taken directly from GitHub's REST API endpoints for pull request review comments. Some field were removed from the JSON schemas for the sake of simplicity, but this is as close to real-world applications as it gets.
Your eagle eye has noticed that both comments and users have an id
field, perfect place to start.
Define decodeId
so that it can decode a JSON object, with an integer id
.
Decode.decodeString decodeId """{id: 10}"""
--> Ok 10
You notice in the example above that not all users have a name.
Define decodeName
so that it can decode a string in the name
field if there is one.
No name? No problem, simply return Nothing
.
This decoder should never fail.
Decode.decodeString decodeName """{"id": 10, "name": "Otto"}"""
--> Ok (Just Otto)
Decode.decodeString decodeName """null"""
--> Ok Nothing
We are pretty much halfway there for decoding users, let's get it done.
Define decodeUser
so that it can decode a JSON user object.
Make sure to use decodeId
and decodeName
in decodeUser
.
Decode.decodeString decodeUser
"""
{
"id": 101,
"login": "octodog",
"avatar_url": "https://githup.com/images/error/octodog_happy.gif",
"site_admin": false
}
"""
--> Ok
--> { id = 101
--> , name = Nothing
--> , login = "octodog"
--> , avatarUrl = "https://githup.com/images/error/octodog_happy.gif
--> , siteAdmin = False
--> }
Some comments in a pull review are standalone, and therefore do not have a pull_request_review_id
.
However, the specification mention that in such a case, there should still be a pull_request_review_id
field, but the value should be null
, the poor man's Nothing
.
Define decodePullRequestReviewId
so that it can decode an integer in the pull_request_review_id
field if there is one.
If there is no integer, there should be a null
, if there is no null
the decoder should fail.
Decode.decodeString decodePullRequestReviewId """{"pull_request_review_id": 3}"""
--> Ok (Just 3)
Decode.decodeString decodePullRequestReviewId """{"pull_request_review_id": null}"""
--> Ok Nothing
Decode.decodeString decodePullRequestReviewId """{"id": 3}"""
--> Err ...
Next is this interesting looking side
field.
In a pull review, you can comment on code that was removed (on the Left
side of the screen) or on code that was added (on the Right
side of the screen).
Define decodeSide
so that it can decode either "LEFT"
or "RIGHT"
in the side
field and match it to its corresponding type variant.
Decode.decodeString decodeSide """{"side": "LEFT"}"""
--> Ok Left
Decode.decodeString decodeSide """{"side": "middle?"}"""
--> Err ...
Each comment has a set of links, which doesn't seem to be consistent across comments.
Define decodeLinks
so that it can decode a collection of URLs in the _links
field.
The names and number of links can vary, make sure you can account for that.
The links themselves are JSON objects that should always have a href
field.
Decode.decodeString decodeLinks """
{
"_links": {
"self": { "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1" },
"html": { "href": "https://githup.com/octodog/Hello-World/pull/1#discussion-diff-1" },
"pull_request": { "href": "https://api.githup.com/repos/octodog/Hello-World/pulls/1" }
}
}
"""
--> Ok
--> (Dict.fromList
--> [ ( "self", "https://api.githup.com/repos/octodog/Hello-World/pulls/comments/1" )
--> , ( "html", "https://githup.com/octodog/Hello-World/pull/1#discussion-diff-1" )
--> , ( "pull_request", "https://api.githup.com/repos/octodog/Hello-World/pulls/1" )
--> ]
--> )
Almost there, let's get it done.
Define decodeComment
and decodeComments
so that they can decode a single JSON comment object or a list of comments respectively.
Make sure to use all of the functions defined so far in decodeComment
, with the exception of decodeName
.
Of course, decodeComments
should use decodeComment
as well.
There is one more feature missing in all of this: being able to send a new comment to the server.
For this, you need achieve the opposite of a decoder, you need to be able to transform an Elm Comment
into a JSON Value
.
Define encodeComment
so that it can encode any valid Comment
.
The order of the fields doesn't matter, as long as the encoder can produce a value that decodeComment
can decode.
Sign up to Exercism to learn and master Elm with 23 concepts, 94 exercises, and real human mentoring, all for free.