"It's time to hang up my nightly suspenders," King said in a message sent to fans via Twitter.
King said he will do occasional specials for CNN. He recently reached his 25-year anniversary and takes pride in a Guinness Book of World Records citation for hosting the longest-running show on the same network in the same time slot.
The long-time radio host was a pioneer in cable television, his desk considered a valued spot to sit for anyone interested in talking to the nation. King's interview style was plain-spoken and critics would suggest occasionally ill-prepared, but he was good at making his guests feel comfortable and ready to talk.
King said he felt no pressure from CNN to leave. He said he began thinking about stepping down as his 25th anniversary week ended earlier this month, on the airplane home after interviewing basketball star LeBron James. During that week, he also spoke to Bill Gates, President Barack Obama and Lady Gaga -- an apt example of the mix that he always sought on his show.
"I said, `I can't top this,"' King said in an interview Tuesday.
"I'm tired of the nightly grind," he said. "I do want to do other things but I want to stay at CNN in some way ... There's a case of great mixed emotions."
King told his staff during a conference call Tuesday that he called "one of the saddest 10 minutes of my life."
As cable news audiences gravitated toward politically pointed shows and newsmakers found many more outlets for interviews, King slipped behind Fox News Channel's Sean Hannity and MSNBC's Rachel Maddow in the ratings. During his interview with Lady Gaga, the 76-year-old broadcast veteran had people wondering if he was really connecting with a pop star a half century his junior.
He's conducted an estimated 50,000 interviews during a 53-year broadcasting career.
He said he always tried to ask short questions, never come in with an agenda and "I left my ego at the door."
"I never learned a thing while I was talking," he said. "That would be my motto."
CNN is in the midst of remaking its prime-time lineup and last week announced that former New York Gov. Eliot Spitzer and conservative columnist Kathleen Parker would co-host an 8 p.m. EDT show on politics and current events.
CNN executives often have said that when "Larry King Live" ends, it won't necessarily be replaced by a similar show. Recent published reports have suggested that "America's Got Talent" judge Piers Morgan could be a candidate for a show in that time slot. CBS' Katie Couric has been considered a potential successor, although that talk has cooled lately.
King said if it was up to him, Ryan Seacrest would be the best choice to fill his shoes.
He dismissed a series of stories this spring questioning his future and speculating about possible successors.
"You can't worry about things you can't control," he said. "I can't control if a newspaper is going to speculate about something or if a blog is going to speculate ... If I let it get me, I'll go nuts. So what I try to do, and I'm not being morbid, I just try to do the best show I can. If it works, it works."
King said he was able to see the baseball all-star games of his sons this weekend. If it was during the week, he'd miss them.
"I'm never going to see these again," he said. "They're not going to repeat themselves. They're 11 and 10. They're not going to be 11 and 10 again."