Chasing a 21st major that would seal a calendar year Grand Slam, Djokovic had difficulty getting into gear against the 99th ranked Brooksby, who for a moment looked capable of derailing the Serb.
For the second consecutive match the world number one came out flat and dropped the opening set, something he will not want to do in his quarter-final when he faces Italian Matteo Berrettini in a rematch of the Wimbledon final which he won in four.
The sixth seeded Italian will mark a big step up in quality for Djokovic, who has not faced anyone ranked inside the top 50, including a qualifier and a wildcard.
The match got off to an extraordinary start as Brooksby took the first set with shocking ease.
Playing for the first time on Arthur Ashe Stadium against one of the all-time greats, Brooksby was unfazed while three-times champion Djokovic sprayed shots helter-skelter, committing 11 unforced errors to just one by the young American. That was the wake-up call Djokovic needed and he broke Brooksby at the first opportunity in the second set, letting out a mighty roar.
Djokovic would win the second but the 20-year-old American made him work for every bit of it, particularly in a grinding fifth game that lasted close to 20 minutes before Brooksby would get the break.
The last American man standing and coming off his first ever five-setter, Brooksby began to fade in the third, going down a double break as Djokovic finally began to dial in his game.
Leading 2-1, Djokovic continued to improve in the fourth and moved clear with a double break and offering zero break point opportunities to his opponent before sealing the win.