These will challenge you more than any dumbbell row. You'll be essentially rowing your bodyweight with one arm.
Don't let your shoulder shrug up, keep your core braced, and pull with control. You'll feel your core work double-time to stabilize the offset load. Your pulling muscles will also contract deeply. To make it easier, simply go flat footed or adjust your angle.
It's a very common problem; almost an epidemic, you could say. This may not be comforting, and it may be something you've heard before, but, "it happens to a lot of guys."