A metermone will help. The challenge of playing with others is that you have to focus some attention on what others are playing, not just your playing. You just gotta keep doing it. It will click - one day you will lock into the groove. It is one of the best feelings!!
I've jammed with friends, played in a high school garage band, and played in a church choir. Even playing the simplest thing is so enjoyable when you all get into the groove.
One question - what's the drummer's experience level? If he is inexperienced playing in a group setting, he may be making adjustments too. You would just have to work on it together.
One more...were you trying to play something that was technically challenging for both of you? If so, you might want to try playing something dead simple that you both know. The point of the exercise would be to find the groove and not have to focus so much on what you are playing. Not only will that help with your sense of timing, but it will boost your morale so you can feel better about what you are doing.