The effect on children is hotly debated.
A UK expert said the level of damage could depend on the individual child.
There is fierce debate over the effect of exposure to television on young children, with increasing evidence to suggest that watching substantial amounts can affect behaviour.
In the US, the American Academy of Pediatrics recommends that children under two years old watch no television, with older children restricted to no more than two hours a day.
The study, from the Johns Hopkins Bloomberg School of Public Health, analysed data for 2,700 children, asking their parents about viewing habits and behaviour at two and five years old.
One in five parents reported that their children watched two hours or more of television daily at both two and five.
This "sustained" exposure was linked to behavioural problems, while parents of children who had watched little television at two, and more than two hours at five were more likely to report problems with social skill development.
Having a television in the child’s bedroom at five years old was linked to behavioural problems, poor social skills and also poor sleep.
However, children who watched more than two hours a day at two years old, but who had reduced their exposure by five, showed no increased risk of any of these problems.
Dr Cynthia Minkowitz, who led the study, said: "It is vital for clinicians to emphasise the importance of reducing television viewing in early childhood among those children with early use."
Dr Richard House, a lecturer in psychotherapy at Roehampton University, and a researcher into the effects of television in young children, said he was not convinced that children who cut back on television would be immune from harm.
He said: "Human behaviour is far more complex than these measures of behaviour and social skills – there may well be some more subtle form of harm that is undetected.
"Every child is different, and I’m very sceptical about the notion that it is appropriate to give families blanket recommendations about the amount of television their children should watch.
"Some children are extremely sensitive to the effects of television."