std::cin with more complex objects

I have a text file which first describes some lines and then describes some colorful lines:

1 2    3  4
5 6    7  8 
9 10   11 12

red    1 0 0    1 2    3 4
green  0 1 0    5 6    7 8
blue   0 0 1    9 10   11 12

The number of lines in each section is unknown at time of execution
I overloaded the std::cin >> operator for these structs:

struct Point { int x, y; }
struct Line { Point a, b; }
struct Color { float r, g, b; std::string name; };
struct ColorfulLine { Line line; Color color; };

(Full example here: http://ift.tt/2c40kJF)
Now I need to iterate over the file using Lines and ColorfulLines:

Line line;                                                     
while(cin >> line) { cout << "We've got a line!\n"; }              

ColorfulLine color_line;                                         
while(cin >> color_line) { cout << "We've got a colorful line!\n"; } 

// actually I'm putting them into std::lists but I skipped this part for simplicity

And here is the problem - the colorful lines are never fetched, i.e. the second loop is not executed.

I have an assumption why it happens but don't know how to fix it:
When std::cin tries to fetch the 4th Line, it fails because instead of a number there's the string "red".
Next, in the second while loop, std::cin tries to read a string (Color.name) but it sees a number, and then fails too.

I tried to put some random word before the ColorfulLines section hoping that when the first loop fails, the second will start reading from the "red" string, but it didn't.

How to fix this?