I went to the sea for the first time, to the dive shop that my friend recommended. I was required to do a checkout dive with an instructor. I thought it was a good idea because it's my first sea diving experience.
I agree. It is/was a very good idea. Ocean diving, with variables such as currents, potentially greater depths, etc. can be quite different from lakes and quarries. That doesn't mean 'better', or even 'harder' necessarily, but different.
We went for a shallow dive (5.5m max depth). I attempted to perform a buddy check with the "BWRAF rule", but got confused because he uses very non-standard gear. I didn't find any buckles, didn't find a weight belt and weights, and both of his regulators were black. Despite this, I didn't ask any questions about his gear and said that everything is ok - it was my first mistake (my inner voice: it's not ok to dive with gear like this! But he is an instructor...). He used a back-inflation BC. I heard about that type but never seen it before. I used normal standard rented gear.
An important part of the pre-dive safety check is gaining an understanding of your buddy's gear configuration, which means asking him about it, if you aren't familiar with it. I would never 'fail' a diver who was unfamiliar with my BP/W, and for whom I had to explain it. I might 'fail' them if they simply did a rote check without asking questions, and overlooked important items.
Our plan was to hover in mid-water and perform several drills: remove and replace the mask, recover the lost regulator, and perform air share. I can do it at the bottom, but I have never done it in mid-water.
You are not alone. Quite a number of divers are unable to manage buoyancy while performing skills, because they were a) never taught to do so, and/or b) haven't progressed to the point of competent multi-tasking. That is a developed skill. Just because you couldn't do it last week, doesn't mean you can't learn it.
We got out of the water and I had a long talk about that crappy dive, but the first that I heard was "You will die! Oh, I meant dive... after taking a refresher course". He was concerned about my buoyancy control skills (or lack of them) and my near-panic reaction. It was too unsafe, so he couldn't allow me to dive.
This is an Instructor 'call'. And, for me, here are the issues, starting with Open Water certification: a) when I certify a diver, or when I just sign a logbook indicating that I have conducted a refresher, my name is now linked to that diver. While I cannot be responsible for all of that diver's future activities, my name being associated with the diver is an indication that I deemed the diver to be competent, at least at the time I affixed my signature; b) I also have a professional obligation to a diver who works with me - if I consider the diver's skills to be inadequate to the point that the diver might conceivably be harmed because of inability to perform skills properly under conditions that s/he is likely to encounter in the near future, I have an obligation to say so, and 'saying so' might be reflected in refusing to certify, or refusing to sign off on a refresher. Frankly, taking YOUR description of your performance (unable to manage buoyancy, unable to perform a regulator recovery skill, unable to control ascent rate, unable to perform an adequate pre-dive safety check) at face value, I might have been inclined to do the same thing, although I certainly would NOT have resorted to the 'You will die!' nonsense. That does NOT mean that I would never 'sign off' on a refresher for you, only that - at that point in time - I believed you needed additional work and skill development.
I have signed up for a refresher course (it starts next week). I'm afraid that he can fail me again! I'm thinking about to give up and throw away my cards now.
Well, I can understand your mindset at the moment. When I was taking technical dive training, I actually 'quit' the course 3-4 times, primarily out of frustration with my own performance. One time, I was in the water with a second trainee, and two Instructors, both of whom were also friends. I had an absolutely horrible training dive - I couldn't do anything right, and got tired of being told, in underwater hand signals, that I was doing things wrong. When we finally exited the water, I took off my rig and drysuit, and packed up my gear without saying anything to the others, then said simply 'That's it for me. I quit. You all go ahead and continue.', and left the quarry. But, each time, after I had a chance to reflect on the situation, I came back with determination to master the skill(s) that I was unable to do at first. And, I did.
I would suggest you work with this particular Instructor again. Yes, as has been noted, it sounds from your description like his communication skills could stand improvement. But, communication requires action from both / all parties. Your job is to a) ask questions if you don't understand something - a skill, a gear configuration, an instruction, etc.), b) ask to try a skill again if it doesn't go well the first time (i.e. keep at it until you get it right), and c) keep working to improve. Now, it may turn out that the 'fit' between you and this Instructor is just not good, in which case going to another shop might be an option.