Ms Rousseff, 62, who had never before held elected office, becomes the country's first woman president.
She promised to "honour the trust" Brazilians had put in her and work to eliminate poverty.
Ms Rousseff enjoyed the full support of President Lula, who is leaving after two terms with record popularity.
Thousands of supporters of the governing Workers Party have taken to the streets across Brazil to celebrate her victory.
The Superior Electoral Court said that with 99% of the votes counted, she had won 56%, against 44% for her rival, Jose Serra of the Social Democratic Party.
This second round of voting was forced after Ms Rousseff fell short of the 50% needed in the 3 October first round. She won 47% to Mr Serra's 33%.
In her victory speech, she said her first priority would be to lift 20 million Brazilians out of poverty.
"I reiterate my fundamental promise: the eradication of poverty", she said.
"We must not rest while there are Brazilians going hungry."
Ms Rousseff said the election of a woman was a sign of Brazil's democratic progress Ms Rousseff is expected to continue the left-leaning policies of President Lula, with emphasis on government efficiency, expanding the role of the state in some sectors such as mining, and upgrading the country's decrepit infrastructure.
She will also oversee a huge expansion of Brazil's oil industry, following the discovery of major offshore fields that should make Brazil one of the world's top 10 oil exporters.
She can count on strengthened majorities for the governing coalition in both houses of Congress to help ease the task of pushing her legislative agenda.
Ms Rousseff's victory owed much to the extraordinary popularity of the outgoing President Lula, who endorsed her as his successor from the start.
Continue reading the main story
BBC News, Sao Paulo
It wasn't the outright first round win President Lula had hoped for, but in the end he has managed to ensure his preferred successor, Dilma Rousseff was elected.
The campaign of Jose Serra was an uphill struggle against a president boasting approval ratings of about 80%. But even though the second round campaign was heated, with many personal attacks and corruption allegations, the candidates didn't differ much in what they had to offer to voters, nor went into great detail over their programmes.
Brazilians have elected Ms Rousseff trusting she will be able to build on President Lula's social and economic achievements. But they do not have a clear idea of the first woman to be elected for the highest office in the country.
She's considered a tough and efficient manager but she hasn't yet shown her political skills, especially when compared to President Lula, who's often described as a master negotiator.
Mr Lula, who has to step down after completing the maximum allowed two consecutive terms, said he would not interfere in her government.
Ms Rousseff will have "to form a government in her own image. I only hope she achieves more than I did", he said after casting his vote.
He added that he would not be attending public victory celebrations because "this is her party".
A former Marxist rebel who was jailed and tortured in 1970-72 for resisting military rule, Ms Rousseff trained as an economist and worked her way up through local and state governments.
She joined President Lula's cabinet as energy minister in 2003-5 and then became his chief of staff.
For Jose Serra, this is the second time he has been defeated in a presidential run-off, after losing to Mr Lula in 2002.
He has congratulated Ms Rousseff and said he hoped she would work for the good of the country.
He said: "I proudly battled the president. To those of us imagining we're defeated: We have only started the real fight."