I thought the almond milk ganache sounded good, too. If it were me trying to bake without butter, I’d use a combination of coconut oil and safflower (or other vegetable) oil, in a ratio that brought the percentage of saturated fat in line with that of butter, probably somewhere around 2/3 coconut and 1/3 safflower. You would also need to add a little almond milk or water to make up for the water content of butter.
La Tourangelle makes good walnut oil and almond oil. I once made a vanilla-almond chiffon with vanilla beans and almond oil that was very, very good. I paired it with coffee whipped cream, which would be out for you, but maybe mocha-almond ganache?