Override Golang JSON Marshalling

We can override Golang JSON marshalling and unmarshalling to encode and decode JSON payload even if the payload does not match the object’s data structure.

Use Case

Suppose you are supporting an API within a workflow where you receive user information, and then you store these records in your database using matching data types

type UserDetails struct {
	Name   string `json:"name"`
	Email  string `json:"email"`
	UserID string `json:"user_id"`
}

Sample payload

{
  "name": "John Doe",
  "email": "[email protected]",
  "user_id": "123456"
}

Change of data type

Now, the upstream sending data to your system has decided to modify their payload, specifically changing the data type of the user ID field from a string to an integer.

{
  "name": "John Doe",
  "email": "[email protected]",
  "user_id": 123456
}

This presents a significant challenge. Firstly, you will need to make changes to your existing API to accommodate the user ID as an integer. Secondly, you will need to perform a database migration to alter the data type of the user ID field. While this task may appear straightforward in certain situations, it becomes complex when dealing with a large number of user records in your database or when foreign key constraints impact multiple schemas.

Override JSON unmarshalling

Fortunately, the user ID value has consistently been in a numeric format, despite being stored as a string data type previously. We can override the JSON unmarshalling process to decode the new user ID field as an integer, allowing us to store the value as an int data type without requiring any modifications to other parts of your system.

To accomplish the overriding of JSON unmarshalling, we implement the UnmarshalJSON function

func (ud *UserDetails) UnmarshalJSON(data []byte) error {
	type UserDetailsJSON UserDetails
	udJSON := struct {
		UserID int `json:"user_id"`
		UserDetailsJSON
	}{}

	if err := json.Unmarshal(data, &udJSON); err != nil {
		return err
	}

	*ud = UserDetails(udJSON.UserDetailsJSON)

	ud.UserID = fmt.Sprintf("%d", udJSON.UserID)

	return nil
}

func main() {
	var payload string = `
{
  "name": "John Doe",
  "email": "[email protected]",
  "user_id": 123456
}`

	details := UserDetails{}
	if err := json.Unmarshal([]byte(payload), &details); err != nil {
		fmt.Printf("error %v", err)
		return
	}

	fmt.Printf("payload unmarshal result: %+v\n", details)
}

The user ID field is correctly populated

payload unmarshal result: {Name:John Doe Email:[email protected] UserID:123456}

Override JSON marshalling for consistency

Assuming that the data type change for the age field will be implemented across all systems and all communication between services will be mandated to use the integer data type, we can conveniently utilize a similar overriding method to maintain clean code within your service. This involves converting the user ID to an integer only during the encoding of the data as a JSON payload.

func (ud UserDetails) MarshalJSON() ([]byte, error) {
	userID, err := strconv.Atoi(ud.UserID)
	if err != nil {
		return nil, err
	}

	type UserDetailsJSON UserDetails
	udJSON := struct {
		UserID int `json:"user_id"`
		UserDetailsJSON
	}{
		UserID:          userID,
		UserDetailsJSON: UserDetailsJSON(ud),
	}

	return json.Marshal(udJSON)
}

func main() {
	details := UserDetails{
		Name:   "John Doe",
		Email:  "[email protected]",
		UserID: "123456",
	}

	result, err := json.Marshal(details)
	if err != nil {
		fmt.Printf("error %v", err)
		return
	}

	fmt.Printf("marshal result: %s\n", result)
}

The resulting payload will have user ID as integer.

marshal result: {"user_id":123456,"name":"John Doe","email":"[email protected]"}

Changing API

In the end, it is still advisable to complete a full migration of your data to ensure consistency with the technical requirements. However, by utilizing this method, you effectively grant yourself additional time to devise a solid migration plan without jeopardizing service availability or risking data corruption.

Making changes to an API within the same version is not advisable, as it requires a considerable amount of time to accomplish. Maintaining backward compatibility should always be a mandatory requirement when working within the same API version, and introducing breaking changes is generally discouraged.