Bit shifting with enums

I have 4 properties and want to shift them in a byte called controlByte. The TypeEnum for example has 4 entries (0 = Unknown, 1 = Red, 2 = Blue, 3 = Green)

• Bit 0-1: TypeEnum
• Bit 2-3: SpeedEnum
• Bit 4: IsActive
• Bit 5: Optional

Code:

 Byte controlByte = 0;
 controlByte  = (byte) (controlByte  | ((int)TypeEnum << 0));
 controlByte  = (byte) (controlByte  | ((int)SpeedEnum  << 2));
 controlByte  = (byte) (controlByte  | (IsActive ? 1 : 0 << 4));

Expected Result:

     | Optional | IsActive | Speed | Type |
 Bit 7          5          4       2      0

I'm not sure if the logic with the enums is correct. But the 3rd line with IsActive doesn't work. The Bit with index 0 is set instead of the Bit with index 4