First Presbyterian Church and the "Old Burying Ground" Cemetery
326 Bloomfield Ave.
Click here for Map / Directions
Although the current church structure was built in 1876, long after the Revolutionary War, the church's history dates back a century further to the Revolutionary War era. What is now Caldwell was settled circa 1740 and was originally part of what was then an area known as Horseneck, which included what was it now Caldwell, Cedar Grove, Essex Fells, Fairfield, North Caldwell, Roseland, Verona, West Caldwell, and parts of Livingston and West Orange.
For several decades, the village had no church of its own. The closest church was in Orange, over ten miles away. In our time, this does not seem like a great distance. However, given the state of local roads and bridges in the mid-1700's and the lack of modern transportation, this meant a half-day's journey.
As the young community that would become Caldwell then lacked a church of its own, traveling ministers would visit the area and conduct services. Since there was then no church structure, they would conduct services in houses and barns, or even outside. Among the visiting ministers was Reverend James Caldwell, the pastor of the Elizabeth Presbyterian Church. Reverend Caldwell was a strong advocate for Independence, who served as both an army chaplain and assistant quartermaster general during the Revolutionary War. (See Reverend James Caldwell Boulder Monument listing below.)
In 1779, an effort was begun under the leadership of Reverend Caldwell to form the First Presbyterian Church in Horseneck. However, it took five years before the first meeting house was built. The Revolutionary War was a factor in the delays. For one thing, it affected the availability of supplies. For another, some of the men were fighting in the war. Twenty Revolutionary War veterans are buried in the cemetery next to the church. (See "Old Burying Ground" Cemetery below)
Work began on the first meeting house in 1782. In 1784, the church association was officially founded. Two years later, the first meeting house was completed. Known as the "Old Parsonage," it was located about two-tenths of a mile south from the current church building, where St. Aloysius Roman Catholic Church is now.
In 1787, this area of Horseneck took the name Caldwell, in honor of Reverend Caldwell. The church took the name The First Presbyterian Church at Caldwell. The community outgrew the first meeting house, and a new church building was begun in 1792 and completed in 1794 at the same location as the current church. That church burned in 1872. The current church building was built on the same site as the previous one, and was completed in 1876. 
As explained above, Reverend Caldwell was instrumental in the formation of this church. In 1787 this area of Horseneck was renamed Caldwell in his honor. This boulder monument honoring him was placed in front of the church in 1924. 
Known as the "Fighting Parson," Reverend Caldwell was a notable figure in New Jersey's Revolutionary War history, with connections to several historic sites. He was a strong advocate for Independence, and he served as both an army chaplain and assistant quartermaster general during the Revolutionary War.
The Revolutionary War experience of James Caldwell and his family is a sad story that gives a picture of how difficult and dangerous life could be in New Jersey during the War. While he was the pastor the First Presbyterian Church in Elizabeth, British forces raided the town of Elizabeth, burning the church, Reverend Caldwell's house, the courthouse, and the Presbyterian Academy school on January 25, 1780.
Reverend Caldwell then moved to Union (then known as Connecticut Farms) with his wife, Hannah, and their nine children. He became minister at Connecticut Farms Presbyterian Church, and the family lived at the parsonage. Sadly, Hannah was killed by a British soldier's bullet fired through a window of the Parsonage on June 7, 1780, during the Battle of Connecticut Farms. The same day the British burned the Connecticut Farms church and Parsonage.
Just sixteen days later, Reverend Caldwell took part in the Battle of Springfield. During the Battle, men ran low on wadding for their muskets. (When firing Revolutionary War era muskets, gunpowder was wrapped in paper. The paper was bit open, and the gunpowder poured in. The paper was then pushed into the musket for use as wadding, to steady the shot.) Reverend Caldwell rode to the Presbyterian Church in Springfield and grabbed a stack of hymn books written by the then well-known hymn writer Isaac Watts. He rode back to the soldiers and threw the hymn books to them to use the pages for wadding, shouting, "Give 'em Watts, boys!"
Caldwell's shouts of "Give em Watts boys!" became the most famous part of the story, and later the subject of a poem by Brette Harte. However, what stands out most to me is the thought of just how personal this fight must have been to Caldwell. Within the previous half year, British troops had burned two of his houses and two churches he preached in. They had also killed his wife, leaving the Caldwell's children without a mother. One can only imagine his feelings that day as he encouraged the American soldiers to shoot British soldiers.
Unfortunately, the tragedies that had followed the Caldwell family throughout the war were not over. Just a year and a half after Hannah's death, James himself was killed by an American sentry in 1781, and their nine children were left orphaned. The sentry, James Morgan, was tried at the Westfield Presbyterian Church, found guilty and then hanged at a site known as Gallows Hill in Westfield.
Both James and Hannah Caldwell are buried in the First Presbyterian Church Cemetery in Elizabeth. 
The cemetery, known as the "Old Burying Ground," was in use circa 1788-1881.  It contains the graves of twenty Revolutionary War veterans. Only ten of these soldiers have gravestones which are still identifiable and readable. Several of them are marked by "Here rests a soldier of the American Revolution" D.A.R. plaques. The other graves have been lost to time.
This is the information for the twenty soldiers. The ten soldiers in italics no longer have visible, readable gravestones. 
Died Dec. 31, 1824, Aged 81
Died Feb. 11, 1841, Age 82
Samuel Crane, Esq
Died Feb. 28, 1811, Age 63
Rev. John Duryea
Died Oct. 2, 1836, Aged 84
Died Nov. 29, 1829, Aged 75
Died Aug. 25, 1807, Aged 70
Dec. 7, 1810
July 23, 1825, Aged 71
Feb. 19, 1841, Aged 88
1757 - Feb. 12, 1847
Rev. Stephen R. Grover
Died June 22, 1836, Aged 77
March 1, 1837, Aged 89
Died June 3, 1841, Aged 84
Enos Martin, Esq.
Died Sept. 1, 1810, Aged 47
Died Aug. 13, 1790, Aged 36
Died Aug. 27, 1797, Aged 60
Apr. 2, 1790, Aged 26
Died March 6, 1801, Aged 63
Captain William Sandford
Died Nov. 8, 1811, Aged 44
Oct. 22, 1836, Aged 76
Reverend Stephen R. Grover 
Among the soldiers buried in the cemetery, two are particularly notable. One is Reverend Stephen R. Grover (July 15, 1758 - June 22, 1836). An obelisk in the cemetery (pictured left), a few feet from the church, pays tribute to him. 
Grover was born in Tolland, CT, and he served as a private in the Revolutionary War in a Connecticut Regiment of the Continental Army. Grover came to New Jersey and became the first pastor of this church in 1787. He would continue as pastor of the church until his death in 1836. During those years, he would serve twice more in military conflicts, in the Whiskey Rebellion in 1794 and in the War of 1812.
Reverend Grover made another notable contribution to history decades after the Revolutionary War era, and indeed after his own death. After Reverend Grover died in 1836, Richard Cleveland became the second pastor of the church. When Cleveland's son was born in 1837, he was named in honor of Stephen Grover.  That child, Stephen Grover Cleveland, would become known simply as Grover Cleveland; he would go on to become president of the United States. He was the only president born in New Jersey.
President Cleveland was born two-tenths of a mile south of the church in a house that served as the church's manse (pastor's residence). That building is now known as the Grover Cleveland Birthplace, and is open for tours. For more information, see their website www.presidentcleveland.org. President Cleveland is buried in Princeton at the Princeton Cemetery, which also contains the graves of several notable Revolutionary War figures.
General William Gould
A memorial post in the cemetery marks the grave of William Gould, who served in the militia during the Revolutionary War. He continued serving in the militia after the Revolutionary War, and he attained the rank of general by the War of 1812. 
Text on the post pays tribute to his service in the Revolutionary War and to his role in the early history of this church: 
"At the age of eighteen, he enlisted in the cause of the Revolution, and was engaged in the Battles of Springfield and Monmouth, he served for many years as a member of the Legislature and Justice of the Peace, and in other offices of Honor and Trust. In the year 1794 he actively assisted in building the House of God, standing near, and being elected one of the first Elders in the Church, he filled the office faithfully and devotedly nearly 70 years."
1. ^ Information in this entry about the history of the church and Horseneck was drawn primarily from the book:
Lynn G. Lockward, A Puritan Heritage, The First Presbyterian Church in Horseneck (Caldwell NJ) (1955)
2. ^ The boulder plaque states that it was "Erected by the New Jersey Society, Sons of the American Revolution, November 24, 1924."
3. ^ For more information and accompanying source notes for the events described in the Caldwell family history in this entry, see the individual town pages linked within the text.
• William S. Stryker, Official Register of the Officers and Men of New Jersey in the Revolutionary War (Trenton: Wm. T. Nicholson & Co., 1872) Available to read at Google Books here
Lists James Caldwell as having served as chaplain of the Third Battalion, under Colonel Elias Dayton (on Page 20),
and having served as an Assistant Quartermaster General (on page 835)
4. ^ Plaque on the fence in front of the cemetery
5. ^ Information about the Revolutionary War soldiers buried here began with:
Lynn G. Lockward, A Puritan Heritage, The First Presbyterian Church in Horseneck (Caldwell NJ) (1955) p. 52
This information was checked against the gravestones in the cemetery to determine the ones that were still there.
I then checked all of this information with:
The Old Burying Ground at The First Presbyterian Church at Caldwell, Essex County, New Jersey - Gravestone Transcription and Mapping Project - Volumes I - IV, Dated October 1989, and July 1990. Prepared by Research & Archeological Management, Inc for The Historical Society of West Caldwell
These four volumes are on file at the First Presbyterian Church at Caldwell office, where I reviewed them. I would like to thank Nancy Felter and Lauren Restaino at the church for granting me access to these volumes, and for being very helpful and friendly while I was there.
6. ^ Biographical and military information about Stephen Grover was drawn from a paper titled Rev. Stephen R. Grover, prepared by Beverly Crifasi for the Historical Society of West Caldwell.
7. ^ This obelisk marks a Grover family plot. The other side of the obelisk marks the graves of Stephen's wife Mary, and their son James.
8. ^ Lynn G. Lockward, A Puritan Heritage, The First Presbyterian Church in Horseneck (Caldwell NJ) (1955) p. 100 - 103, 150 - 151
9. ^ An article contributed to the Montclair Patch by the Historical Society of West Caldwell, "War of 1812 Flag on View This Memorial Day" (May 26, 2012) describes Gould's service after the Revolutionary War and into the War of 1812.
10. ^ This memorial post marks a Gould family plot. The other side of the memorial marks the graves of William's wife, Mahetable, and their daughters Harriet Henrietta Gould and Charlotte Harrison.
• "At the age of eighteen..." quoted text appears on the grave post, underneath what is visible in the photo.