Theoretically, that is exactly what you should do. That way you know it is that particular thing that is causing the problem instead of one of the 3 or 5 ingredients that are different than the dog food that was fine. But some dogs don’t show sensitivity to the fresh food where they do to the same ingredient in kibble. In kibble you don’t know exactly what part of the chicken is in there , for example.
BTW, after you try the sweet potato, if you find he reacts to it, you need to give him a couple weeks of just the food you know he is OK with before you try the next possible allergen to make sure the last one and its histamines are completely out of his system.
