Getting access to schools - and then keeping it - tested director Jo Abel and her team.

It's day 28 of an eight-month shoot. I'm standing in the playground of a north London secondary school and 900 students are bowling past with hoods up and scowls on. Two strapping 16 year olds stroll up to the camera. One looks into the lens and says: “Suck my balls.”

No thanks, we reply. I can tell it's going to take a bit of time to become invisible.

Alice, the producer and I are leading a slightly schizoid existence. One week we are at the Cheltenham Ladies' College surrounded by Cheltenham ladies in their glorious, green uniforms, the next we are hanging out in said playground.

We are making two films at the same time for a new BBC4 series called My New Best Friend. A separate team - director Sasha Djurkovic and producer Lucy Cohen - are battling the rain in the Scottish highlands making the third film in the series at Malaig High School.

Two months earlier I got a call from Ed Coulthard at Blast! Films: “Fancy making a couple of films about kids leaving primary school and starting secondary school? The focus of the series is to hear only the children's voices. Lovely commission from [BBC commissioning editor for documentaries] Richard Klein.”

Ed's inspiration came from the daily walk to school with his own children, who talked constantly about who their best friends were - and who those friends would be when they went to senior school. Alice and I jumped at the chance.

David Brindley, the development producer, had spent months carefully negotiating access with Cheltenham Ladies' College. I go up to meet the headmistress, Vicky Tuck, thinking it's a done deal. “Well,” she said. “Would you like to tell me about the films you have made?” My mind goes completely blank.

Gaining access into schools is a nightmare. The schools don't trust you; they want to know exactly what you want to do. What's more, you are asking to film 11 year olds they barely know - their new intake. There are also numerous ethical and legal dilemmas inherent in working with this age group.

My (only) ace is that in a previous life I was a teacher and about seven years ago I worked with director Paddy Wivell, again for Blast! Films, on two Channel 4 films, Boys and Girls. We had spent a year in Kingsmead Primary School in Hackney, working closely with Prash Naik, the lawyer at C4, who guided us through the many difficult ethical and legal challenges of working with 10 and 11 year olds.

With Cheltenham under our belt we contacted hundreds of London schools, the vast majority of which said no. Islington Arts and Media School in Finsbury Park was interested “in principle”. With the support of the headteacher, Richard Ewen, and the chair of governors, the school finally agree to let us in. Then we had to find the kids.

Choosing children who can articulate their journeys for 60 minutes seemed an impossible task. We visited children who lived on estates in Finsbury Park and girls who lived in mansions in the Cotswolds. It's instinct that draws you to some and not others. The boys, Azad and Demian, who feature in the London school, made us laugh. And all four of the Cheltenham girls - Annabelle, Lydia, Daisy and Nanae - were extraordinarily articulate and loved talking.

In many ways the biggest challenge of a long-form observational documentary is keeping access going while focusing on the stories you are following. In both schools we had good relationships with staff and ingratiated ourselves by doing extra filming for the drama and English departments, as well as hanging round in the caretaker's office at Islington Arts and Media and bringing in lattes each day for Dan Bethell, the head of year seven.

Secondary schools must be one of the noisiest places to film so sound was a challenge. We used endless amounts of gaffer tape to secure the radio mics on the children and to try to reduce the constant rustle of their clothing. We also used a two-way mixer which gave us greater range and reduced (some) of the ear-splitting distortion when filming in the playground.

We did become part of the furniture in both schools and in the children's lives. And we learned in making the films that when you are 11 years old and a Cheltenham girl, a boy from an estate in Finsbury Park, or a child from the Scottish Highlands, what you look for and value in your friendships is the same - loyalty and trust.

Although the children all voiced their anxieties before they started at secondary school, once they got there they were more excited than frightened. They were sad to leave the familiarity of primary school but glad to be moving on into a more grown-up and independent world.

Having such a long filming period allowed us the luxury of building real relationships and seeing what happened as the children made this massive transition. I'd love to go back in five years' time and see what has happened to them all.

My New Best Friend is a Blast! Films production for BBC4. It airs on Wednesday 21 May at 9pm.

Jo Abel: My tricks of the trade
Retain a sense of humour at all times.

Book a series of appointments with an osteopath in advance - you know your back will give out carrying gear, filming and driving up and down motorways.

Always carry gaffer tape - especially when working with children!

When you get it wrong, just admit it.

To keep access, carry on talking even when you'd rather not.