Samsung’s claim that the new Galaxy Z Flip uses “Ultra Thin Glass” sounded like a true breakthrough when the foldable phone was announced last week. Until now, foldable screens have used plastic displays, which can be easily scratched with even a fingernail. The Z Flip making the switch to glass, however thin it might be, had us hopeful that it would hold up better to long-term use.

But it seems that might not be the case.

Zack Nelson has gotten his hands on Samsung’s second attempt at a foldable phone, and the results of his JerryRigEverything durability test do not inspire confidence.

The Z Flip’s display starts showing permanent marks and scratches far earlier than actual glass would. Part of the tried and true JerryRigEverything test is putting phones through a gauntlet of Mohs hardness picks to test when the display glass starts showing damage. If you’ve watched Zack’s videos before, you’ve likely heard that modern smartphones have “scratches starting at a level 6, with deeper grooves at a level 7.”

The Z Flip starts picking up damage at level 2 and more significantly at 3, which is on par with the plastic screens of the Galaxy Fold and more recent Motorola Razr. “This screen is in no way scratch resistant whatsoever,” Nelson says near the end of the video. At the end of the clip, he begins poking holes in the screen that make the OLED panel go on the fritz — but there’s no sign of any glass fracturing.

At the unveiling of the Galaxy Z Flip, Samsung was categorical that it was glass. My colleague Dieter Bohn pressed Samsung representatives for more information on who Samsung’s partner is for manufacturing it, how it was made to bend, and what processes were applied to the glass. At the time, Samsung declined to comment on all those questions. But the company offered a response when we asked for comment on this story.

Galaxy Z Flip features an Infinity Flex Display with Samsung’s Ultra Thin Glass (UTG) to deliver a sleek, premium look and offer an immersive viewing experience,” a spokesperson said by email. “Samsung’s first-of-its-kind UTG technology is different from other Galaxy flagship devices. While the display does bend, it should be handled with care. Also, Galaxy Z Flip has a protective layer on top of the UTG similar to Galaxy Fold.

Is Samsung trying to say that we’re only seeing scratches on the outer protective layer? Those look pretty deep and permanent, but it’s possible. This is the layer Samsung is talking about:

We asked Samsung if it planned to offer a screen replacement service for the Z Flip as it did with the Galaxy Fold. It will. Z Flip buyers can get a one-time screen replacement for $119, Samsung says. In addition, you can also get a specialized screen protector designed for the Z Flip for free one time. Here’s the company’s statement on that:

As part of Premier Service, we will offer a one-time free application of a screen protector for the Z Flip at select UBIF [U Break I Fix], Samsung branded locations or by sending it to Samsung Premier Service via mail,” the company added. “The screen protector will be applied by a specialist with the proper equipment to align and apply it. The program is rolling out soon.

Nelson thinks Samsung could be using a hybrid plastic polymer (with microscopic bits of glass mixed in) so it can advertise this display as “glass.” But if it really can be scratched this easily, what seemed like a significant selling point for the $1,380 Galaxy Z Flip is starting to look like overblown marketing.