This feature is now complete for Google and Microsoft calendars. You will need to disconnect / reconnect your Google calendar to have it sync the color correctly.

Unfortunately, we couldn't get this working with Apple calendar as their API doesn't return the color in the calendar information.