Best is what your county and city uses for their roads.
Clay is not an organic material, loam is. Organic means that their is plant matter decomposing in it. Clay is a silt type material that has dozens of different names because it varies depending on what it's made up of. Here, red clay is the most desirable fill material because it compacts extremely well, it sheds water when compacted and it holds the weight of a building easily. Some people might confuse what happens when water puddles on the surface of clay soils as creating mud, when in fact it's holding water and there isn't anyplace for the water to drain to.
Build up your roads with the red clay so that there are ditches on either side of the road to allow water to run off of the road and away. Compact the clay. For my personal home, I put a full load of dirt in my front bucket and then go over the clay with my front tires over and over again. Your city and county will use a vibratory roller to compact it in small lifts of about 4 inches. Just dump it, spread it, then compact it until you get it up to where you want it.
Road base gravel consists of a variety of different sized rock that has been crushed. The crushed part is very important. Those jagged edges allow the rock to compact with each other to form a solid surface that will shed water. It needs to be at least 4 inches thick, with thicker being better. I watched Oak Island and saw that they put down well over a foot for their roads, which was very impressive. The rock also needs a lot of water to set up. At first, this will make the rock mushy, but once it dries out, it will be super hard. The best way to get enough water on it is a heavy rain. Ideally you don't want to drive over it right after that first big rain since so much water will soak into it, but then after a few days, it will harden up and you will see a huge difference.
The type of rock used and the name given to it varies around the country. This is why you need to find out what your city and county use when they build their roads. Here in East Texas, we call it "Roadbase" or "Cliche" which is crushed limestone.