Place potatoes in a saucepan, and cover with water. Bring to a boil. Reduce heat; simmer 10 minutes or until tender. Drain. Cool and peel. Cut potatoes into 1/2-inch cubes. Place potatoes in a large bowl; sprinkle with vinegar and oil. Add celery, onion, pickle relish, and eggs; toss gently.
Combine mayonnaise, mustard, salt, and pepper. Spoon mayonnaise mixture over potato mixture; toss gently to coat. Cover and chill 1 to 24 hours.