Category Archives: Device drivers

Device drivers, Insider stuff, Recent Activity, WED Blog, Windows 11, Windows Update

ThinkBook Hybrid Fails 24H2 Upgrade

Leave a comment

I can’t say I’m surprised, but I can confess to mild bepuzzlement. Over the weekend, WU decided that a new test laptop — a Lenovo ThinkBook Plus G5  — was ready for 24H2. Alas, for a variety of good reasons that ThinkBook hybrid fails 24H2 upgrade and goes down in flames. Indeed, I had to use the WinRE “Uninstall upgrade” option for the first time ever to bring that machine back to life.

Why ThinkBook Hybrid Fails 24H2 Upgrade

Of course, when I checked the machine this morning, I hadn’t interfered with WU, so it was already attempting this doomed upgrade again. You can see it’s at 25% complete in the lead-in graphic. I foiled that by restarting the PC, then pausing updates for 7 days in the WU controls, as you can see here:

While I’m troubleshooting, I suspended updates.

What the Panther Logs Have to Say

There are two primary logs worth checking into if an upgrade fails to complete successfully — namely:


C:\$Windows.~BT\Sources\panther\setupact.log
C:\$Windows.~BT\Sources\panther\miglog.xml

These are liberally bestrewn with errors of all kinds including device drivers and storage items. The ThinkBook’s “hybrid drive” bridges between native Windows storage on the deck and native Android storage on the tablet. It takes the form of something called a Hybrid Tab in File Explorer (see next screencap). I believe it’s very likely involved in my snafu.

Because it uses a special driver and customized software, I’m betting that the Hybrid Tab is the source of storage and driver issues.

Copilot tells me that if I upgrade from bootable media, the ThinkBook may survive the upgrade process intact. Right now, I think it’s losing the Hybrid Tab scaffolding amidst the post-GUI reboots during the serious parts of OS install. That sounds like it’s worth a try. But first, I’ll make a fresh image backup and be ready to restore same should things go south again.

Stay tuned! This could get interesting…

Facebooklinkedin
Facebooklinkedin
Device drivers, Insider stuff, Recent Activity, Troubleshooting, Updates, WED Blog, Windows 11

BIOS Update Demands Cable Switch

Leave a comment

Whoa: this time, things got just a little bit TOO interesting. I’ve got a Lenovo P360 Ultra ThinkStation on loan, and a BIOS update came through today (to version S0JKT2AA). But when I would install the update, the usual BIOS flash screens did not come up after a reboot. It wasn’t until I swapped the graphics cable from the full-size DP to full-size DP port, to a full-size DP (monitor) to mini DP (PC) that the splash screen showed up at boot, and the BIOS flash ran through to completion. Thus, the BIOS update demands cable switch to succeed. Go figure!

How Did I Figure Out That BIOS Update Demands Cable Switch

By watching the post-reboot behavior on-screen, I realized it wasn’t showing me what it was supposed to. Basically, the screen stayed black post-restart until the lock screen for Windows 11 appeared. I knew I was supposed to see the boot-up splash screen (which reads “Lenovo” in white letters on a black background on this device). But instead: nada.

So on a whim, I brought down the video & power cables box from atop my bookshelves. Then, I grabbed a full-size DisplayPort to mini-DP cable and used it to replace the full-size DP to full-size DP I was currently using. Immediately thereafter, I got a splash screen and the BIOS update started processing. It took a while, but it eventually ground through to a successful update.

What About those Intel Graphics?

The next item of business was to get the built-in Intel graphics (UHD Graphics 770) updated. After a handful of failed attempts to get the Lenovo version to run, I visited the Intel DSA (Driver & Support Assistant) and installed that version instead. It worked. You can see the results for my final — and entirely welcome — update check using the Lenovo Commercial Vantage tool as the lead-in graphic above.

That was a wild ride. But indeed, that’s the way things go in Windows-World far too often, based on my current level of interest vs. fatigue. Today, fatigue wins out. Sigh.

Facebooklinkedin
Facebooklinkedin
Device drivers, Insider stuff, Recent Activity, Updates, WED Blog, Windows 11

KB5043145 Throws Interesting Stopcode

4 Comments

Here’s one I’ve not run into before: Stopcode 0XEA. It shows up on BSODs as THREAD_STUCK_IN_DEVICE_DRIVER. The intro screencap show results running that stopcode against the MS error code lookup tool. Basically, it says a device driver thread is stuck in an infinite loop. (See the MS Bug Check 0xEA page for further deets.) Apparently, it’s showing that optional CU KB5043145 throws interesting stopcode and BSOD/GSOD  on some PCs. Notably, says WindowsLatest, that includes some 2022 and 2024 Asus laptops.

If KB5043145 Throws Interesting Stopcode, Then What?

If that happens to any of your PCs, you’ll need to boot to WinRE on bootable media, and use the “Uninstall Update ” item in its Advanced Options menu to uninstall it from your Windows Image. When a PC won’t boot because the image is damaged, that’s pretty much the only repair that works, short of a clean Windows (re)install.

Alas, this is eerily reminiscent of the July 19 Crowstrike update, which took down 8+million Windows PCs. Fortunately, it doesn’t seem to be anywhere near as widespread nor impactful as that incident turned out. That said, Windows users should be aware that this optional CU could force recovery and repair to undo. Fortunately, such updates do not typically affect production environments, where update get tested and vetted long before they get scheduled for some update window.

But gosh, it seems like we’ve run into rather more problematic updates that is typical in 2024. FWIW, the update hasn’t caused any trouble on those test machines here at Chez Tittel where I can make it run. Even so, it’s a great reminder to be careful out there in Windows-World…

Facebooklinkedin
Facebooklinkedin
Device drivers, Insider stuff, Recent Activity, Troubleshooting, WED Blog

Morning Black Screen Recalls Pending Reboot

Leave a comment

On September 10, NVIDIA release its Game-Ready driver, version 561.09. At its conclusion it asked for a reboot. “Oh yeah,” I thought, “I’ll do that later.” It’s happening a LOT later than I planned, nearly 8 days on. If you look at the uptime info in the lead-in graphic you’ll see I’ve somehow managed no reboots since then. But, for the last two days this PC’s monitors have stayed dark when I’ve tried to wake it up first thing in the morning. Alas, that morning black screen recalls pending reboot, which I apparently MUST do (soon).

Note: I’ve been able to bring the desktop back from the black screen state on each of the past two days by striking CTRL-ALT-DEL at the keyboard, then canceling out of the Security Options screen that pops up. Good thing to know, in case this ever happens to you.

How Morning Black Screen Recalls Pending Reboot

Normally, when I click a mouse button or hit a keyboard key when my PC is sleeping, it starts right up. Both yesterday and today, though, I get black screens on both monitors with no cursor. Experience informs me that this is 95+% likely caused by a graphics driver issue. And as I think about it, I dimly recall installing 561.09 last week, then never following up with a reboot. If you do the math on the uptime field from WinFetch in the lead-in graphic, it was last updated on September 9,  around 3:09 PM (thanks timeanddate). Thus, it hasn’t been updated since the GPU driver got updated.

I’ve also noticed graphics running a bit slower and jerkier lately, too. It all adds up: I should’ve remembered to reboot the same day I updated NVIDIA graphics driver. But it may be too late to go back, but it’s not too late to reboot right now. And sure enough, when I do, no more black screens on startup, nor after waking from sleep (which I forced from Power > Shutdown > Sleep through the Start Menu to check).

Go figure. I should know better. This not-so-gentle reminder does the trick to help me remember this time. Isn’t that just the way things sometimes go in Windows-World?

Facebooklinkedin
Facebooklinkedin
Backup/Restore, Device drivers, Insider stuff, Recent Activity, USB devices, WED Blog

Considering USB4 External Media

Leave a comment

It’s a classic trade-off in more ways than one: cost versus speed. I’m prepping for an AskWoody story about external media on Windows PCs. For me, the big trade-off when considering USB4 external media is higher prices for higher performance. “How much higher?” you ask: that’s what I’m in the process of figuring out right now.

Whole Device Chain Counts When Considering USB4 External Media

Every step in the device chain counts when going for the speediest external Windows media. The starting point from the PC end is the USB port itself. Ideally, it should be USB4 or Thunderbolt 3/4, and support 40 Gbps throughput. Next comes the cable: it should be labeled USB4, Thunderbolt 3 or 4, or 40 Gbps. Next comes the storage device. For me, that mostly means an enclosure housing an NVMe SSD. That enclosure should be USB4 or Thunderbolt 3/4, and the NVMe should be Gen 3 (PCIe x3) or higher.

At every step you pay more to attain the current pinnacle of performance. (I’ve not yet seen any 80 Gbps devices, but they’re coming. Copilot tells me Intel’s 14th Gen HX-series mobile CPUs “are starting to support this technology. “) A quick search at Amazon tells me you can’t buy USB5 cables, docks, and so forth yet. My best guess: we’re looking well into 2025 before it goes mainstream.

Right now, the jump from USB 3.2 Gen 2 (10 Gbps) to USB4 (and TB 3/4 equivalents) is getting cheaper, but still costs. You’ll pay US$46 for the cheapest USB4 M.2 enclosures right now (more like US$75 and up for other options). That’s double the cost — or more — of USB 3.2 Gen 1 devices (UASP: see below). Cables cost US$2 to $10 more for faster varieties, which isn’t too punitive. You can’t take advantage of anything faster than Gen 3 NVMes. Thus, you can buy 1 TB for US$55-80, and 2 TB for US$93-130 or so.

The “big spring” comes from the cost of either buying (for laptops and so forth) or installing (for desktops with open PCIe slots, and ASUS is the only vendor I can find who makes one for US$126) to gain a USB4 40 Gbps port to plug into. My testing so far shows this DOES make a difference, and often offers better performance than older and rarer Thunderbolt 3 or 4 capable USB-C ports.

For Me, Backup Is the Killer App

I’m always messing with PCs, so I need to back up frequently in case I shoot myself in the foot and have to replace a mangled installation. It happens to me at least 1-2 times a week in my testing and research, so this is no joke. I find the cost of USB4 external storage worthwhile because it drops the time to make a complete image backup into the 2-4 minute range. It takes anywhere from 7-24 minutes to back up to UASP-capable external storage. This equates to USB 3.2 Gen 2 10 Gbps capability. It shows up with max read/write speeds in Cystaldiskmark in a range from 1000-1100 MBps.

If you look at the lead-in graphic, which comes from NirSoft’s USBdeview, you can see it references the UASPStore.sys driver and service. I’ve actually found this to be a clearer way to recognize when a USB 3.x port can provide somewhat higher speeds. If your USB 3.x ports are older (and slower) they’ll usually show a USBstor.sys driver instead (and max speeds in CrystalDiskMark in the 400-500 MBps range).

You pays your money, and you takes your chances. That’s how things go with external USB-attached Windows storage — and much else in life!

 

Facebooklinkedin
Facebooklinkedin
Device drivers, Insider stuff, Recent Activity, WED Blog, Windows 10, Windows 11

Reinstall Solves Nvidia Driver Issue

Leave a comment

This morning, I noticed a new GeForce driver for the RTX 3070 Ti on my production system via the NVIDIA app. Perforce I installed same. Then the screen flashing started: on and off, and on and off, and … you get the idea. I’ve been here before, but usually when an old driver needs replacing. I tried to roll back (but the option didn’t light up) so I used the REINSTALL facility you see at the upper right of the lead-in graphic. I’m happy to report that such a reinstall solves NVIDIA driver issue. The blinking/flashing is fixed. Good-oh!

Why Reinstall Solves Nvidia Driver Issue

When a graphics driver misbehaves  — as in going into blink mode — it’s usually because the driver itself is messed up. Or it might be that communications between the driver and the device aren’t working properly. Either way, something needs to change.

Ordinarily I’d use the rollback facility on the Driver tab for the RTX 3070 Ti to revert back to the previous version. But, as you can see in the next screencap, the “Roll Back Driver” button is greyed out. Alas, that means there’s nothing to roll back to. Looks like the new NVIDIA app doesn’t keep old driver versions around. What to do?

Because it’s greyed out, I can’t use Roll Back Driver.

Fortunately, as you can see in the lead-in graphic, the NVIDIA app offers a REINSTALL option for the current driver. Because the installed version was blinking I figured I had nothing to lose by giving REINSTALL a try. So I did just that.  Just to make sure everything got replaced, I also checked the installer’s Clean Installation option. And fortunately for me, it worked!

The desktop is back to normal, and the blinking is behind me now. All’s well that ends well in Windows-World, especially when it comes to misbehaving graphics drivers.

Facebooklinkedin
Facebooklinkedin
Copilot+ PCs, Device drivers, Insider stuff, Networking, Recent Activity, WED Blog, Windows 11

T14s Mystery Ethernet Controller

Leave a comment

As part of my usual shakedown for new PCs here at Chez Tittel, I went poking around inside Device Manager on the newly-arrived ThinkPad T14s. As it is a Snapdragon X Elite (Copilot+ PC) its networking interfaces come from Qualcomm. The laptop has no RJ-45 port (nor really, any room for one). And yet, a device named “Ethernet Controller” shows up in DevMgr in the always-puzzling “Other devices” category (see lead-in graphic). This has me chasing down this T14s mystery Ethernet controller.

Where’s the T14s Mystery Ethernet Controller?

Nowhere, as far as I can tell. I tried to find a driver for it, and couldn’t. I checked the chip and laptop documentation. I finally “got clue” when I looked in the User Manual under the heading “Connect to the wired Ethernet,” which reads as follows (helpful diagram included):

To connect your computer to a local network, you need a Lenovo USB-C to Ethernet Adapter. Lenovo USBC to Ethernet Adapter is available as an option and shipped with some computer models. You can purchase one from Lenovo at https://www.lenovo.com/accessories.

Shoot, Lenovo’s available model handles up to 2.5 GbE but requires separate purchase for an eminently reasonable US$32 or so. What I still don’t know is if the T14s chassis does include built-in wired Ethernet capability, albeit bereft of an actual link-up (which the illustrated item above nicely remedies). I’ve seen this on other Lenovo laptops (including the X380 Yoga which uses a proprietary laptop port rather than USB-C) so I’m still wondering…

Just Because I Can, Doesn’t Mean I Will

My local wired Ethernet still runs Gbe (1.0 speed, in other words). I’m not yet ready to upgrade my infrastructure to the higher speed. I can use my Starlink GbE dongle (USB 3.0 to RJ-45) instead if I must. That said, the Wi-Fi 7 interface on the T14s runs like a champ on my 802.11ac (Wi-Fi 6) network, so it’s all good right now.

It’s interesting that an Ethernet device shows up at all in DevMgr. I’ve never seen this before on Wi-Fi only laptops. But then, I’m still learning the ins and outs of Snapdragon X ARM-based Windows PCs. Good fun!

Facebooklinkedin
Facebooklinkedin
Copilot+ PCs, Device drivers, Insider stuff, Recent Activity, WED Blog, Windows 11

Dxdiag Shows WDDM Version Info

Leave a comment

Just recently, I’ve seen coverage that proclaims a new Windows Display Driver Model (aka WDDM) is available as version 3.2. That said, it’s currently supported only in Copilot+ PCs running Windows 11 24H2. Since I happen to have one, I wanted to check to see if I needed to update that manually, or if WU/install had done so already. That’s how I learned that the Direct X Diagnostic Tool, aka Dxdiag shows WDDM version info on Windows PCs. You can see the output that confirms 3.2 is running as the lead-in graphic above.

Confirmed: Dxdiag Shows WDDM Version Info

Just for grins, I also ran the diagnostic tool on my Lenovo ThinkPad P16 Gen 1 Mobile Workstation. (Hint: to do that, enter Winkey+R to open the run box, type “dxdiag,” then click Enter.) It’s running Windows 11 23H2 (Build 22631.3958). According to the great ElevenForum.com tutorial from uber-tut-master Shawn Brink “Find WDDM Version in Windows 11” it should be running version 3.2. Yet dxdiag shows 3.1 as you can see here:This is more in keeping with the WindowsLatest story that clued me into this version upgrade’s availability. And indeed, Copilot confirms that it only runs on version 24H2, not on 23H2 (as the afore-linked ElevenForum tutorial asserts). I think I’ll shoot Brink a message to ask for his opinion on this apparent discrepancy. Stay tuned!

But Wait…There’s More!

For further grins, I checked one of my Insider Preview Canary Channel PCs right now, and observed that:

  • This version is part of the 24H2 family (Build 26257.5000)
  • It is not a Copilot+ PC (Intel 8th Gen/Tiger Lake i7-1180G7)
  • It also runs WDDM 3.2, as you can see here:

To me, this indicates pretty unequivocally that 24H2 is the way to get WDDM 3.2. Thus, it shows up on older PCs running 24H2 in the Insider Preview family tree, as well as newer ones with 24H2 pre-installed (right now, that’s only Copilot+ PCs). Go figure!

Facebooklinkedin
Facebooklinkedin
Device drivers, Insider stuff, Recent Activity, WED Blog, Windows 10, Windows 11

NVIDIA Driver Forces POPCNT Support

2 Comments

Hey! We’ve known it was coming for some time now. In February 2024, Tom’s Hardware reported: “Microsoft’s PopCnt restriction appears to be unbreakable.” This CPU instruction is part of the SSE4.2 Instruction set for Intel processors. It’s SSE4.2a for AMD CPUs, and also present in Snapdragon X as well. Indeed it’s been part of the computing landscape since 2008 for Intel, circa 2009-2010 for AMD, and since inception for ARM (including Copilot+ PCs). But now, with the introduction of version 555, NVIDIA driver forces POPCNT support. How so? Check the lead-in graphic. Windows 10 and 11 systems that lack POPCNT support will BSOD if they install this latest GPU driver version. Ouch!

Sussing Out NVIDIA Driver Forces POPCNT Support

You can read about this in the NVIDIA release notes for 555 (source of the lead-in graphic). It’s entitled “EOL Windows driver support for older CPUs without POPCNT instruction.” That pretty much says it all. The note recommends using Sysinternals Coreinfo to check for POPCNT support. (Indeed, it worked to show such support on the Lenovo ThinkPad T14s Copilot+ loaner PC.)

But on Intel and AMD PCs, I found it easier to use Franck DeLattre’s excellent CPU-Z tool to look for the SSE4.2 (or SSE4.2a) instruction set instead. The ARM version of CPU-Z is less than helpful in providing POPCNT info because it doesn’t natively map to any ARM instruction set. Here’s what comes up on my ancient i7 SkyLake desktop (running Windows 10 for lack of CPU support).

Note the “Instructions” entry that reads SSE4.2 (bottom of “Processor” block): that’s what we’re looking for, when using this tool to check for POPCNT support on Intel CPUs. WindowsLatest reports that CoreInfo doesn’t work reliably on all systems, and recommends other CPU check tools besides CPU-Z. Be sure to check it out for those details if one or the other of these tools doesn’t cut it on your rig.

Older PCs: Avoid NVIDIA 555

The real takeaway from this news is that owners of older PCs whose CPUs don’t support POPCNT should avoid updating to NVIDIA driver versions number 555 or greater. Otherwise, they’ll have to boot into WinRE, and use DISM at the command line to uninstall that driver to end a BSOD loop on that machine. You’ve been warned! I’d be inclined to take this as a early indicator that a hardware refresh should be in your future, too…

 

Facebooklinkedin
Facebooklinkedin
Cool Tools, Device drivers, Insider stuff, Remote Desktop (RDP), Security, Thoughts & concerns, WED Blog

Getting Past Crowdstruck Requires Access

3 Comments

Last Friday (July 19), cybersecurity firm Crowdstrike pushed an update to its threat sensors. Ultimately, that ended up with over 8 million Windows PC unable to boot, stuck on a BSOD for invalid references in a kernel-mode driver. Behind the scenes, all kinds of companies from hospitals, to government agencies, to airlines, and more, found themselves unable to use updates machines after a post-update reboot. What really caused the heartburn? Getting past Crowdstruck requires access to affected machines on a one-at-a-time basis.

If you look at the BSOD screencap at the head of this blog post, you’ll see a driver named csagent.sys. This is the CrowdStrike Agent driver which runs at kernel mode by design. That ensures it can’t be easily accessed or tampered with by hackers. But when something runs as a kernel mode driver it must be rigorously and thoroughly tested and vetted, or it can crash any PC on which it runs. Errors, in short, cannot be tolerated. Oops!

Why Getting Past Crowdstruck Requires Access

Part of the Crowdstrike software run as a Windows kernel-mode driver. That means it has the same level of access as privileged parts of the OS itself. If any of this code throws an error — as Crowdstrike has publicly admitted its update did — Windows crashes itself. That’s by design,  out of an abundance of caution to avoid loss of data or other damage to affected systems.

Here’s where things get interesting. Windows can’t boot and run until the offending driver is removed. In turn, the affected PCs must boot into safe mode or a recovery image. Either can operate on the damaged Windows image, remove the bad driver, and stand Windows back up again. This is easy when admins or IT pros have physical access to affected PCs. Indeed, Copilot recommends using the “three strikes” method to get into Windows recovery. (Three consecutive boot failures autoomatically triggers Windows alternate boot.) Then, using WinRE (or Windows itself in safe mode, from the Advanced Boot Options), repairs can go forward.

The problem is that many, if not virtually all, of the affected machines stayed down, stuck in a “boot loop.” They remained that way because their operators DIDN’T have physical access to those PCs. I’ll bet that most of them had to be teleoperated through a KVM device that can work around PC  problems that extend all the way down to the hardware level (outside the scope of normal remote access and RDP). This kind of thing doesn’t scale well, either, so it takes time to work through hundreds to thousands of remote PCs (think of the PC behind the counter at AA or Delta, where the gate or ticket agent is completely clueless about boot-level Windows repairs).

An “Interesting” Problem, Indeed!

Far too many cybersecurity and IT pros found themselves in the grip of the old Chinese curse (“May you live in interesting times”) after the *291* driver for Crowdstrike  tried to run on Friday. Organizations that prepare and drill for these kinds of outages were doubtless at an advantage in already knowing how to broker and run boot repairs remotely. I can only imagine the hair-pulling that went on at other outfits less well-equipped to handle this outage.

Here’s a moral to ponder for those who run remote Windows PCs where physical access is impossible, difficult or impractical: Can your remote management infrastructure and automation work with a Windows PC that’s not booting, and won’t boot until it’s restarted in some special way? If your answer is “yes,” you’re probably over the Crowdstruck hump already. If your answer is “no,” you’ll probably make that a top priority as soon as you can kick-start and repair all remaining affected Windows nodes. In the meantime, my deepest sympathies…

Facebooklinkedin
Facebooklinkedin