As early as 1915, the Fern Hill community was served by a series of library stations and reading rooms sponsored by local businesses.

In 1931, members of the Fern Hill Parent-Teacher Association wrote to the Library Board requesting a branch library for their community. The depression of the 1930s prevented any attention to Fern Hill's request, but in 1939 the idea was revived by the Greater Library Committee, with Nels Bjarke, a Scandinavian immigrant, as Chairman.

In 1941, the property on which the library is now situated was purchased by the Greater Library Committee and the deeds presented to the Library Board. The land, at the corner of South 84th Street and Yakima Avenue, came to be known as Byrd Square, after George Washington Byrd, who settled in the Fern Hill area in 1866. Byrd later platted the area and deeded a large portion of his land to the town to be used as a school ground. Byrd Square is part of that deeded area.

In November 1948, the Tacoma City Council approved the construction of a branch library in the Fern Hill area and, in February 1950, the 2,900 square foot library opened to the public. The new library, built at a cost of $40,000, was designed by Silas Nelsen, and held 8,000 books and other library materials. One room of the new facility was designated as The Pioneer Room and was used for community meetings.

The building remained virtually unchanged until a small addition was added in 1961 as part of a six-year Capital Improvement Program. Another addition to the building, a larger community meeting room, was completed in 1978.

The $15.8 million Bond Project program passed by Tacoma voters in 1984 provided for the construction of the current 7,996 square foot library (nearly 2,300 square feet larger than the former building).