How can I correctly implement this operation with averaging unsigned chars - C++

Consider the following, seemingly simple, problem, that I've come across in solving assignments in CV involving masks and implementing AA in ray tracing.

An color in the RGB format has three channels, each channel represented by an unsigned char, hence 0 <= channel value <= 255. For reasons of symmetry, let's consider a single channel. I want to sum up channel values over a certain set of points, then average and round that (not just truncate) and store back into an unsigned char. Note that in all the problems, the resulting averaged value is guaranteed to be in the range [0, 255], it's an invariant of the algorithm.

This looks like an easy task, but I'm concerned with the following:

p.1. The most important: the type choice, see pseudocode below for which types I need. More precisely, I'm getting a narrowing conversion error, which turns into an error with -pedantic-errors set, and I don't want to simply wave it away.

p.2. The idea of going from a wider type that can accumulate many uchar's, back to the uchar type. I'm confident that the algorithm is guaranteed to produce a value in the uchar range, but is it enough? Should I worry about this?

Pseudocode (cv::Vec3b is a fixed size 3x1 array of unsigned char in OpenCV which iI use for image storing and output):

// this returns a value in the range [0, 255]
cv::Vec3b processing_func(const point&) { ... }

accum_type acc_r = 0, acc_g = 0 , acc_b = 0;
for (point p: point_set) {
    const cv::Vec3b color = processing_func(p);
    acc_r += color[0]; acc_g += color[1]; acc_b += color[2];
}
cv::Vec3b result_color{roundf(acc_r / (floating_type)set_size),
                       roundf(acc_g / (floating_type)set_size),
                       roundf(acc_b / (floating_type)set_size)};
// hello narrowing conversion ^^^

Refer to p.1, what should the accum_type and floating_type be (the latter required to divide correctly without truncation), or would you maybe code it differently?

Edit after the first comment: set_size is an integer above 0, it can be hardcoded if required, but in general is bound to change. It's e.g. AA factor, squared; or mask size, and so on.