ATLANTA (AP) — Joe Biden declared he “should not have been so cavalier” on Friday after he told a prominent black radio host that African Americans who back President Donald Trump “ain’t black.”

The presumptive Democratic presidential nominee quickly moved to address the fallout from his remark, which was interpreted by some as presuming black Americans would vote for him. In a call with the U.S. Black Chamber of Commerce that was added to his public schedule, Biden said he would never “take the African American community for granted.”

“I shouldn’t have been such a wise guy,” Biden said. “No one should have to vote for any party based on their race or religion or background.”

That was an acknowledgement of the stinging criticism he received in response to his comments, which he made earlier in the day on “The Breakfast Club,” a radio program that is popular in the black community.

The rebukes included allies of Trump’s re-election campaign and some activists who warned that Biden must still court black voters, even if African Americans overwhelmingly oppose the president.

Near the end of Biden’s appearance on the radio program, host Charlamagne Tha God pressed him on reports that he is considering Minnesota Sen. Amy Klobuchar, who is white, to be his vice presidential running mate. The host told Biden that black voters “saved your political life in the primaries” and “have things they want from you.”

Biden said that “I guarantee you there are multiple black women being considered. Multiple.”

A Biden aide then sought to end the interview, prompting the host to say, “You can’t do that to black media.”

Biden responded, “I do that to black media and white media,” and said his wife needed to use the television studio.

He then added: “If you’ve got a problem figuring out whether you’re for me or for Trump, then you ain’t black.”

South Carolina Sen. Tim Scott, a Trump supporter and the Senate’s sole black Republican, said he was “shocked and surprised” by Biden’s remarks.