The Lenox Avenue Gang was started in the early 1900s by Horowitz as an independent group of around twenty members. It consisted mostly of pickpockets and burglars, under Jack Zelig's Eastman Gang. Mainly operating around 125th Street, the gang generally committed muggings and robberies although they were occasionally hired out for murder by Zelig. Under Horowitz's leadership, the gang produced many of the top criminals of the early century, including Jacob Seidenschner, Louis Rosenberg, and Francesco Cirofisi. Prominent New York police detective Val O'Farrell called Cirofisi "one of the toughest men in the world", and police suspected Cirofisi of at least six homicides. However, they were unable to charge him due to lack of evidence. An associate of the gang, Cirofisi's girlfriend Dutch Sadie was known to carry a butcher knife in her clothing, and she assisted Cirofisi in several muggings.
While successful in its early years, the gangs downfall began when they were hired to murder gambler Herman Rosenthal, a suspected police informant. Horowitz, Seidenshner, Rosenberg, and Cirofisi drove to the Metropole Hotel on July 16, 1912 and, calling out to Rosenthal, shot him several times as he appeared on the street before fleeing the scene. The gangsters were seen by dozens of witnesses and quickly arrested. When questioned, they told police that they were hired by NYPD Lieutenant Charles Becker, who had ordered Rosenthal's death after he had informed on Becker.
Gyp the Blood, Jacob Seidenschner, Lefty Louis Rosenberg, and Francesco Cirofisi were charged with the murder of Rosenthal, sentenced to the electric chair, and executed on April 13, 1914. With the loss of their leaders, the Lenox Avenue Gang disbanded entirely within several months.