"I may leave fast" if progress does not seem possible, Trump said in western Pennsylvania. Trump said he believes North Korea wants to make peace and that, "I think it's time," Reuters reported.
A time and place to meet has not yet been set, although the meeting is supposed to happen by the end of May.
"Who knows what's going to happen?" said Trump, who added that if the meeting takes place, "I may leave fast or we may sit down and make the greatest deal for the world."
Trump made the shocking decision on Thursday to meet with Kim after the North Korean leader's invitation was relayed by a South Korean delegation who visited the White House. The move abruptly reversed decades of US policy aimed at preventing North Korea from developing nuclear weapons and ballistic missiles.
Earlier in Washington, Trump sought to rally international support for a potential summit, saying North Korea had agreed to not conduct another missile test until after proposed meetings had taken place.
"North Korea has not conducted a Missile Test since November 28, 2017 and has promised not to do so through our meetings. I believe they will honor that commitment!" Trump wrote on Twitter.
Trump's comments aligned with what a South Korean official stated on Thursday about the possible talks. South Korean officials said this week that Trump had agreed to an invitation from Kim to meet by May.
The White House has been under fire for agreeing to talks and responded to the criticism on Friday by warning that no summit would occur unless Pyongyang took "concrete actions" over its nuclear program.
Trump also took to Twitter on Saturday to characterize the leaders of China and Japan as supportive of the potential dialogue, yet did little to clear up confusion over the preconditions and timing of any talks.
Trump and Kim prompted jitters around the world last year as they exchanged bellicose remarks over the North's attempts to develop a nuclear weapon capable of hitting the United States. Pyongyang has pursued its nuclear program in defiance of United Nations Security Council resolutions.
Tensions eased around last month's Winter Olympics in South Korea, laying the groundwork for what would be the biggest foreign policy gamble for Trump since he took office in January 2017.
The head of South Korea's National Security Office, Chung Eui-yong, said in Washington on Thursday he briefed Trump about a meeting South Korean officials held with Kim this week. He said Kim had "committed to denuclearization" and to suspending nuclear and missile tests.
Trump's sudden decision to meet with Kim stunned even people in his own administration.
"The expectation is that the talks would lead to a discussion around a conclusion that we're ready to engage in negotiations," a senior State Department official said on Friday.
Some US officials and experts worry North Korea could buy time to build up and refine its nuclear arsenal if it drags out talks with Washington.