For a given focal grid cell, the terrain ruggedness index (TRI) is calculated by taking the square root of the average of the squared difference between the focal cell's elevation and the elevations of the 8 surrounding cells, or $$\sqrt(\sum_{i = 1}^{8}(m_i - m_0)^2 / 8)$$ where \(m_0\) is the elevation of the focal cell and \(m_i\) is the elevation of the ith grid cell.
References
Riley, S.J., DeGloria, S.D., and Elliot, R. 1999. A terrain ruggedness index that quantifies topographic heterogeneity. Intermountain Journal of Sciences 5:23-27.