Many Caribbean coral reefs are undergoing a phase shift from coral to macroalgal dominance. Understanding the processes driving changes in algal abundance and community structure requires clarification of the relative effects of top-down (e.g., herbivory) and bottom-up processes (e.g., light, temperature, and nutrients). To date, a number of studies have examined the relative effects of grazing versus nutrification but interactions between herbivory and natural, seasonal fluctuations in temperature and light have not been investigated. This study considered the dynamics of three Caribbean macroalgal species [Lobophora variegata (Lamouroux), Dictyota pulchella (Hörnig and Schnetter), and Halimeda opuntia (Linnaeus)] and algal turf. A field experiment was established to measure species-specific algal dynamics (changes in abundance) over 13 months in the presence and absence of herbivory. Both herbivory and seasonal changes were important processes controlling macroalgal and turf abundance. Water temperature and light had a key role on D. pulchella; this species’ abundance significantly increased in the summer, when water temperature and light were the highest, and decreased during winter. Surprisingly, herbivory did not seem to control D. pulchella directly. However, herbivory was the most important process controlling the abundance of L. variegata, H. opuntia, and turf. The abundance of both algal species was correlated with seasonal changes in the environment, but was depleted outside cages throughout the year. The abundance of H. opuntia was positively correlated with temperature and light, but there was no statistical interaction between drivers. The statistical interaction between temperature and light was significant for the abundance of L. variegata and turf, but algal abundance declined as both factors increased. Overall, macroalgal and turf cover were mainly controlled by herbivory, while community structure (which species contributed to the overall cover) was largely influenced by seasonal changes in temperature and light.