Besides the period costumes and sets, one of my favorite things about the movie is that every now and then a character will name-drop a real-life actor or filmmaker from that era ("in Gable's last picture..." or "Did Selznick produce that one?..."). It becomes something of a running gag that I really got a kick out of.
It's really interesting to see actors like Kristen Stewart and Blake Lively appear in a Woody Allen film. They're not bad actors, it's just that some of their previous work has been geared to a younger audience. Jesse Eisenberg was in To Rome with Love.
Another Hollywood-themed movie I saw was the original 1937 version of A Star is Born, with Janet Gaynor and Fredric March. It's also set in the same time period, the 1930s. A starstruck Janet moves from her farm town to Hollywood to become an actor, and finds love and support from March. They eventually get married and are one of the most famous couples in Hollywood (but of course she is more famous -- a star on the rise, and he's on the way down). The strangest part of the film was the trailer-in-the-woods honeymoon sequence, which seemed a bit out of place and questionable destination for a honeymoon for two big name stars.
I liked the scene where a drunken March makes a speech in front of the crowd; he does something similar in front of a crowd in The Best Years of Our Lives during a dinner scene.
If you've seen any of the A Star is Born movies then you know what happens at the end, but I won't spoil it if you don't know what happens....just that it ends on a sad note.
I think these two movies - Cafe Society and 1937's A Star is Born - would make for an interesting double feature centering around 1930s Hollywood.