Categories
go json

JSON and dealing with unexported fields

85

Is there a technical reason why unexported fields are not included by encoding/json? If not and it is an arbitrary decision could there be an additional back door option (say ‘+’) to include even though unexported?

Requiring client code to export to get this functionality feels unfortunate, especially if lower case is providing encapsulation or the decision to marshal structures comes much later than design of them.

How are people dealing with this? Just export everything?

Also, doesn’t exporting field names make it difficult to follow suggested idioms. I think if a struct X has field Y, you can not have an accessor method Y(). If you want to provide interface access to Y you have to come up with a new name for the getter and no matter what you’ll get something un-idiomatic according to http://golang.org/doc/effective_go.html#Getters

    115

    There is a technical reason. The json library does not have the power to view fields using reflect unless they are exported. A package can only view the unexported fields of types within its own package

    In order to deal with your problem, what you can do is make an unexported type with exported fields. Json will unmarshal into an unexported type if passed to it without a problem but it would not show up in the API docs. You can then make an exported type that embeds the unexported type. This exported type would then need methods to implement the json.Marshaler and json.Unmarshaler interfaces.

    Note: all code is untested and may not even compile.

    type jsonData struct {
        Field1 string
        Field2 string
    }
    
    type JsonData struct {
        jsonData
    }
    
    // Implement json.Unmarshaller
    func (d *JsonData) UnmarshalJSON(b []byte) error {
        return json.Unmarshal(b, &d.jsonData)
    }
    
    // Getter
    func (d *JsonData) Field1() string {
        return d.jsonData.Field1
    }
    

    5

    • For the record, I had to Unmarshall using “json.Unmarshal(b, &d.jsonData)”. Did I do something wrong, or is that expected?

      – Derek

      Mar 28, 2013 at 20:05


    • @Derek, thanks, I updated my answer. As I said the code was untested. I also apparently also forgot a return statement in my UnmarshalJSON() method. I fixed this as well.

      Mar 29, 2013 at 8:00

    • 10

      I’m a bit late to the show, but… while the above works, Field1 and Field2 are exported. You can read and write Field1 and Field2 of JsonData (with a capital J) outside of that package. So, while this is cool in theory, it doesn’t actually do anything different than exporting both the type and the fields.

      Feb 28, 2017 at 3:17

    • 1

      @davidjosepha you’re correct as the answer was originally shown although someone would have to be looking into your source code to know that Field1 and Field2 are present since godoc wouldn’t list the unexported struct jsonData. However, this issue can be fixed by not using an Anonymous field in the JsonData struct, and having a named field which is unexported. This then requires that Field1 and Field2 are accessed through the named, unexported field. I edited the answer to use this methodology.

      – ddrake12

      May 15, 2018 at 18:42

    • @davidjosepha You can’t read and write Field1 of JsonData.jsonData outside of that package directly because JsonData has a method Field1 that “override” the inner field. go.dev/play/p/LvEI_Dexc6S If you want to protect Field2, you have to add a Field2() method of course.

      – Cirelli94

      Mar 28 at 8:59


    71

    Stephen’s answer is complete. As an aside, if all you really want is lowercase keys in your json, you can manually specify the key name as follows:

    type Whatever struct {
        SomeField int `json:"some_field"`
    }
    

    In that way, marshaling a Whatever produces the key “some_field” for the field SomeField (instead of having “SomeField” in your json).

    If you’re dead-set on keeping unexported fields, you can also implement the json.Marshaler interface by defining a method with the signature MarshalJSON() ([]byte, error). One way to do this is to use a struct literal that simply has exported versions of the unexported fields, like this:

    type Whatever struct {
        someField int
    }
    
    func (w Whatever) MarshalJSON() ([]byte, error) {
        return json.Marshal(struct{
            SomeField int `json:"some_field"`
        }{
            SomeField: w.someField,
        })
    }
    

    That can be a bit cumbersome, so you can also use a map[string]interface{} if you prefer:

    func (w Whatever) MarshalJSON() ([]byte, error) {
        return json.Marshal(map[string]interface{}{
            "some_field": w.SomeField,
        })
    }
    

    However it should be noted that marshaling interface{} has some caveats and can do things like marshal uint64 to a float, causing a loss of precision. (all code untested)

    3

    • Technically, ALL numbers in Javascript are floats.

      May 14, 2013 at 14:21

    • 27

      JSON is not JavaScript. The JSON spec merely states which characters are acceptable, not which numeric ranges are valid. A number like 876234958273645982736459827346598237465923847561203947812435968234659827346 is still valid in JSON, even if it can’t be understood by JavaScript. For a real world example, the Twitter API represents tweet IDs as 64bit unsigned integers, which is not valid in JavaScript, but is valid JSON.

      – jorelli

      May 14, 2013 at 19:19

    • 2

      Currently, all true, but still the name is JSON, which historically stands for “JavaScript Object Notation”. One of those tiny suprising bits that make you want to rename it to NJSON (NewJSON or rather NotJavaScript) 🙂

      Jul 13, 2014 at 20:28