Cook potatoes, whole and unpeeled, in water just until tender when pierced with fork. Drain, then set aside until cool enough to handle. Potatoes can be cooked a day ahead and refrigerated until ready to make the salad.
Slice unpeeled potatoes into slices that are a little over 1/4″ thick.
In a large frying pan, fry bacon until crisp. Remove bacon from pan and drain on paper towels. Reserve 2-3 tablespoons bacon fat in the pan. Discard remainder of fat.
Over medium heat, cook onion in bacon fat until translucent.
Stir in the vinegar, sugar, water, salt, celery salt, mustard powder, and black pepper. Stir to combine.
Add potatoes to mixtures and toss to distribute ingredients evenly. Heat over medium heat, stirring as needed, until potatoes are warmed throughout. Serve while warm. Can also be served at room temperature.