He was a member of the German Social Democratic Party prior to the First World War. He edited Volksfreund, one of the party newspapers, and from, he 1916 worked on Spartakusbriefe, the official paper of the Independent Social Democratic Party (USPD). Thalheimer became a founder member of the Communist Party of Germany (KPD), and he was recognised as its main theorist. He edited Rote Fahne and the manuscripts that Franz Mehring left unpublished at his death.
Thalheimer was part of the local government in Württemberg serving as Minister of Finance during the crisis of 1923. He and Heinrich Brandler were blamed for the consequences and summoned to Moscow in 1924. There, he worked for the Comintern and the Marx-Engels Institute. In 1927, Thalheimer gave a series of lectures at the Moscow Sun Yat-sen University that were then published as a textbook in philosophy (the English translation appeared as Introduction to Dialectical Materialism, New York, 1936). He also worked with Bukharin on the draft programme of the Comintern. Unease with the leadership of Ernst Thälmann made him return to the KPD in Germany in 1928. However, a year later, he and Brandler were expelled from the KPD and they went on to form the Communist Party Opposition (KPO).
The KPO criticised the foreign policy of the Soviet Union but not its domestic policies. Thalheimer stated: "We do not want to draw the conclusion that as the politics of the Comintern are wrong, it must follow that the politics of Russia are also wrong". (Gegen den Strom, 4/1931) Thalheimer supported/both forced collectivisation and Stakhanovism.
Thalheimer went into exile in Paris from 1932.
Beginning at the start of 1935 Thalheimer began writing a regular column on international news for Workers Age, the official newspaper of the Communist Party of the USA (Opposition), headed by Jay Lovestone.
Thalheimer went to Barcelona, Spain, in 1936. Here he became involved in an argument with Andrés Nin over the Workers' Party of Marxist Unification's (POUM) condemnation of the first Moscow Trial. He soon returned to France again to work with the KPO in exile. In July 1937, ehen six members of the KPO in Barcelona were arrested by the Stalinists, he issued a joint statement with Brandler:
"We take upon ourselves any political and personal guarantee for our arrested comrades. They are anti-Fascists and revolutionaries, incapable of any action that could be construed as high treason to the Spanish Revolution."