Before Max Scherzer, Juan Soto, and Stephen Strasburg led their baseball team to the World Series, the Washington Nationals were made up of federal government workers. And they played not in a 41,418-seat stadium next to the Anacostia River, but on a patch of grass right outside of the White House.

Baseball in D.C. has a long, storied history that predates even the Washington Senators and Griffith Stadium. It was 160 years ago when the nation’s capital got its first baseball team.

The National Base Ball Club of Washington D.C., otherwise known as the “Nationals,” formed in 1859. The players were mostly government clerks from the Treasury, the IRS, and auditors’ offices.

Their first games were played somewhere on Capitol Hill, but in 1865, they moved to a field in front of the White House (where the Ellipse is today). They competed in a tournament there, which attracted some pretty high profile fans.

“Six thousand fans watched, including President Andrew Johnson,” says Fred Frommer, author of You Gotta Have Heart: A History of Washington Baseball. “But the Nationals lost both of their games, first to Philadelphia 87 to 12 and, then, the second to Brooklyn 34 to 19. People complain about the ball being juiced now, but, you know, come on.”

Prior to the Civil War, baseball was already on its way to becoming the national pastime. A shot-off of cricket and an English game called “rounders,” the sport was officially first played in New York in 1846 before migrating south. By the late 1850s, the game had found its way to D.C..

According to Frommer’s book, two teams called Washington home by 1859: The Nationals and the Potomacs. They were rivals, often playing in front of a crowd of fellow federal workers.

Fearful that things could get rowdy, the teams imposed fines on what was considered improper behavior by the players. There was a 10 cent fine for any profane language, 25 cents for disputing an umpire’s call, and another 10 cents for even expressing an opinion about a close call before the ump could decide his verdict.

“There was this vision of baseball as a gentlemen, upper-middle-class game,” says Frommer, “But that didn’t last long. It became a rougher sport by the late 19th century.”

During the Civil War, the game began to take on national prominence. An early reference to baseball can be found in an 1860 political cartoon featuring then-presidential candidate Abraham Lincoln holding a long club-like bat and ball with his foot stepping on home plate.

There were a number of D.C.-based teams by the late 1860s playing in a number of different semi-pro leagues, which included ball club names like the Olympic, Capital, Interior, the Union, Empire, and the Cicero Debating Society. These teams were made up of only white players.

There were several African-American teams in D.C. as well, most prominently the Washington Mutuals and the Washington Alerts. It was around this time when the unofficial “color line” was drawn in the sport.

While white and black teams rarely played each other, they sometimes used the same fields. In 1867, the Philadelpia Pythians took on the Washington Alerts in such an anticipated match-up that the white Philadelphia Athletics offered up their field for the game.

While the Washington newspapers wouldn’t cover this game, outlets in New York and Philadelphia did. One New York headline highlighted a high profile fan. It read “Frederick Douglass Sees a Colored Game” and explained that the well-known orator was there to see his son, who played for the Alerts.

By 1870, the game was changing, particularly in terms of who was playing and coming to the games. “Baseball was in flux,” says Frommer, “It wasn’t as popular as was it was going to become in the 20th century, but it was catching on.”

A poolroom owner by the name of Mike Scanlon built a 500-seat ballpark along S Street NW, which is thought to have extended two blocks between 15th and 17th Streets. With easy access to the trolley, it was a popular neighborhood spot. It had an unfortunate drainage problem, so muddy conditions often plagued the field. Today, it’s the site of several apartment buildings and an off-leash dog park.

In the mid 1880s, the Washington Nationals of the newly formed National League moved to a new home on the eastern side of North Capitol Street. While the park was officially called Capital Park, it was also known as Swampoodle Grounds. It was named after the neighborhood, so-called by recent Irish immigrants because of the marshy, wet soil that encouraged a portmanteau of “swamp” and “puddle.” The park held 6,000 fans, but the team only played there for three seasons.

Two decades later this site became Union Station, with the National Postal Museum also now sitting on a portion of the land that was once a ballpark.

Like in the present day, ball players were well-known and liked back then, too. In Frommer’s book, there’s an anecdote about how visiting players would stay at the Willard and sit out in front of the hotel on Pennsylvania Avenue in armchairs chewing tobacco while talking with fans.

“Players had a certain stature,” says Frommer. “And there was an openness that may not be there as much today.”

Perhaps the most famous player that played for the Nationals back then was William Hoy, who is believed to be the first deaf player in baseball history. Fans, teammates, and media all called him “Dummy,” and he even referred to himself by this nickname, according to the Society of American Baseball Research. He was a star, particularly for his skill in stealing bases.

Hoy used his status as a ballplayer to raise awareness for the deaf community. Some historians even credit the ballplayer for getting the umpires to use hand signals for balls, strikes, safe, and out, rather than just calling them out loud. Hoy’s playing ability and community service are still honored today. About a mile away from his old stomping grounds, Gallaudet University renamed one of its baseball diamonds after him in 2011.

The turn of the century also meant a new team and league for a Washington baseball franchise. In 1901, the American League formed, with the Washington Nationals as one of its founding teams.

For the next six decades, to separate itself from the previous Nationals, they were most often known as the Washington Senators. In 1960, the team moved to Minnesosta and became the Twins.

A few months later, Washington was awarded an expansion team to make up for their loss and this team became officially known as the Washington Senators as well. However, a decade later in 1971, they too moved—this time to Texas.

And, then, in 2005, the Washington Nationals returned to D.C. But unlike their predecessors, they do not play in front of the White House.