Re: Questions about probing WLUNs for UFS power management

I looked through the code again and re-thought the design.

1) We need to control the suspend/resume order between normal LUNs and
WLUNs, so the power-management logic has to live in the UFS driver
(WLUN should suspend only after all normal LUNs are suspended).
Because of this, I think we should call
EVENTHANDLER_REGISTER(suspend/resume) in the UFS driver.
2) Since the power-management logic is in the UFS driver, the UFS
driver should build the START STOP UNIT CCB and call xpt_action().
3) If suspend/resume(power-management logic) is handled inside the UFS
driver, the WLUN periph would only probe WLUNs.

So we need to choose:
(1) do we implement a WLUN periph and let SCSI XPT own the cam_ed, or
(2) do we have the UFS driver call xpt_alloc_device() and own the
cam_ed directly?

Given that power management is implemented in the UFS SIM/driver,
option (2) seems more suitable to me. What do you think?

Jaeyoon

On Thu, Oct 2, 2025 at 5:13=E2=80=AFPM Jaeyoon Choi <[email protected]> w=
rote:
>
> Hi Warner,
>
> I=E2=80=99m not fully sure this is the best path yet, but I looked more i=
nto option (1).
> Below are the questions I had and the answers I found.
>
> Q. Are the semantics of START STOP UNIT (SSU) in the UFS spec the same
> as in SBC-5?
> A. Yes. SBC-5 defines SSU in a flexible way, and the UFS spec defines
> its SSU usage to be compliant with SBC-5.
>
> Q. Where is SSU used in (linux) UFS driver?
> A. Power management. During suspend the driver sends 'UFS-Sleep (POWER
> CONDITION =3D 0x2)' or 'UFS-PowerDown (0x3)', and during resume the
> driver sends 'Active (0x1)'.
>
> Q. Can we reuse an existing periph driver?
> A. No. Existing periph drivers aren=E2=80=99t suitable for WLUNs. The pas=
s
> thru device also doesn't implement suspend/resume.
>
> Q. How do we register handlers for suspend/resume in FreeBSD?
> A. A periph driver can register handlers at init time with
> EVENTHANDLER_REGISTER(power_suspend, ...) and
> EVENTHANDLER_REGISTER(power_resume, ...).
>
> Now that those questions have been clarified, I fully understand your com=
ments.
>
> At this point, it seems we need to create a periph driver like
> `scsi_wlun.c` for attaching WLUNs, and implement suspend/resume
> function within it.
> In suspend/resume, we should issue the SSU command.
>
> Planned tasks:
> 1) Define a `PIM_WLUN` flag so only drivers that want WLUNs discover them=
.
> 2) Use `REPORT LUNS` with `SELECT REPORT` =3D 0x01 to obtain the WLUN lis=
t first.
> 3) Add a `CAM_GET_WLUN()` macro to parse the WLUN list.
> 4) When `CAM_GET_WLUN()` finds a match, automatically attach the WLUN per=
iph.
> 5) In `scsi_wlun.c` 'wluninit()', register power handlers
> (EVENTHANDLER_REGISTER(power_suspend, wlunsuspend, ...),
> EVENTHANDLER_REGISTER(power_resume, wlunresume, ...)).
> 6) In `wlunsuspend() / wlunresume()`, issue SSU commands to the UFS Devic=
e WLUN.
> 7) The order between LUN periph (da) and WLUN periph must be guaranteed.
>    (Ensure that the WLUN SSU is issued last during suspension and
> first during resumption.)
>
> It's Korean Thanksgiving day until October 10. I'll take a short break
> and then start prototyping this right after.
> If I've misunderstood anything, please let me know.
>
> Jaeyoon
>
> On Wed, Oct 1, 2025 at 12:11=E2=80=AFPM Jaeyoon Choi <[email protected]=
> wrote:
> >
> > > It's easy enough to add the enumeration for (1) to scsi_xpt.c It's ju=
st a few more states in the probe state machine (which is a bit complex, bu=
t not terribly so). If we treat them as separate LUNs, then a lot of the in=
frastructure would just work. The key, though, is how do we keep scsi_da, e=
tc from attaching to these new nodes. And would we have generic scsi_foo.c =
drivers for them (so the devices would get their own periph drivers) or whe=
ther there's a higher-level device that would know it could use these devic=
es if it finds them and it wouldn't have a specific specialized kernel driv=
e for them, but instead use a generalized pass thru device.
> >
> > That's correct. Probing the WLUN isn't difficult. And being able to
> > use the SCSI infrastructure is a huge advantage.
> > However, as you mentioned, having the peripheral driver (scsi_da?)
> > attached to the WLUN node seems like it could be a problem.
> >
> > >
> > > I guess a lot of that would depend where things like power management=
 is happening. If it's in the UFS sim, that would suggest a different desig=
n than if this was some super generic thing that would respond to generic e=
vents that maybe the SIMs generate or maybe somebody else.
> >
> > Refer to Section 7.4.2 =E2=80=9CPower Management Command: START STOP UN=
IT=E2=80=9D in
> > the UFS 2.1 specification. The START STOP UNIT command is used to
> > change the power mode.
> > The SBC-5 spec also defines the SSU's POWER CONDITION field. (SBC-5
> > 5.31 =E2=80=9CSTART STOP UNIT command=E2=80=9D)
> > The specs seem similar, but I need to study further to understand
> > whether UFS's power management approach is a super generic thing.
> >
> > I'll come back after reading more of the spec and code.
> >
> > Thanks,
> > Jaeyoon
> >
> > On Mon, Sep 29, 2025 at 6:48=E2=80=AFAM Warner Losh <[email protected]> wr=
ote:
> > >
> > > Please excuse the tardy response
> > >
> > > On Tue, Sep 23, 2025 at 8:32=E2=80=AFPM Jaeyoon Choi <jaeyoon@freebsd=
.org> wrote:
> > >>
> > >> Hello freebsd-scsi,
> > >>
> > >> I=E2=80=99m trying to implement Well-known LUNs (WLUNs) for the UFS =
(Universal
> > >> Flash Storage, ufshci(4)) driver, but the current SCSI XPT doesn't p=
robe
> > >> WLUNs, so I have a few questions.
> > >>
> > >> First, Well-known LUNs are defined in SCSI SAM-5. In UFS, 'REPORT LU=
NS',
> > >> 'UFS Device', 'RPMB', and 'BOOT' LUNs are treated as WLUNs.
> > >> (SAM-5 4.7.5.1 "Well-known logical unit addressing",
> > >> UFS 2.1 spec 10.8.5 "10.8.5 Well Known Logical Unit Defined in UFS")
> > >>
> > >> Because a UFS device contains multiple LUNs, power management requir=
es
> > >> sending a START STOP UNIT(SSU) command to the 'UFS Device' WLUN. (If=
 SSU
> > >> is sent to a non-WLUN (normal) LUN, the device power mode is not cha=
nged
> > >> and only that LUN becomes disabled.)
> > >> (UFS 2.1 spec 7.4.2 "Power Management Command: START STOP UNIT")
> > >>
> > >> In FreeBSD, the SCSI XPT logic that parses LUNs checks the 'Simple l=
ogical
> > >> unit addressing format' (SAM-5 Table 26) and the 8-byte 'Extended lo=
gical
> > >> unit addressing format' (SAM-5 Table 35), but it does not check the
> > >> 'Well-known logical unit extended addressing format' (SAM-5 Table 37=
).
> > >> In the scsi_xpt.c code below, CAM_GET_LUN reads Extended LUNs and
> > >> CAM_GET_SIMPLE_LUN reads Simple LUNs:
> > >>
> > >> ```
> > >> sys/cam/scsi/scsi_xpt.c line:2114
> > >> static void
> > >> scsi_scan_bus(struct cam_periph *periph, union ccb *request_ccb)
> > >> ...
> > >>             while (scan_info->lunindex[target_id] < nluns) {
> > >>                 if (scan_info->cpi->hba_misc & PIM_EXTLUNS) {
> > >>                     CAM_GET_LUN(target->luns,
> > >>                         scan_info->lunindex[target_id],
> > >>                         lun_id);
> > >>                     break;
> > >>                 }
> > >>
> > >>                 if (CAM_CAN_GET_SIMPLE_LUN(target->luns,
> > >>                     scan_info->lunindex[target_id])) {
> > >>                     CAM_GET_SIMPLE_LUN(target->luns,
> > >>                         scan_info->lunindex[target_id],
> > >>                         lun_id);
> > >>                     break;
> > >>                 }
> > >>
> > >>                 scan_info->lunindex[target_id]++;
> > >>             }
> > >> ```
> > >>
> > >> Also, we can obtain the number of WLUNs via the REPORT LUNS SCSI com=
mand.
> > >> To retrieve WLUNs, the SELECT REPORT field must be set to 0x01, but
> > >> SCSI XPT currently uses 0x00 to obtain only normal LUNs.
> > >> (UFS 2.1 spec 11.3.12.2 "Report LUNS Command Select Report Field Val=
ues")
> > >>
> > >> ```
> > >> sys/cam/scsi/scsi_xpt.c line:827
> > >> static void
> > >> probestart(struct cam_periph *periph, union ccb *start_ccb)
> > >> ...
> > >>     case PROBE_REPORT_LUNS:
> > >> ...
> > >>         scsi_report_luns(csio, 5, probedone, MSG_SIMPLE_Q_TAG,
> > >>             RPL_REPORT_DEFAULT, rp, periph->path->target->rpl_size,
> > >>             SSD_FULL_SIZE, 60000);
> > >> ```
> > >> ```
> > >> sys/cam/scsi/scsi_all.h line:3020
> > >> struct scsi_report_luns
> > >> {
> > >>     uint8_t opcode;
> > >>     uint8_t reserved1;
> > >> #define RPL_REPORT_DEFAULT     0x00
> > >> #define RPL_REPORT_WELLKNOWN   0x01
> > >> ```
> > >>
> > >> Looking at the Linux code, Linux also does not check WLUNs at the SC=
SI
> > >> layer. Therefore, the Linux UFS driver explicitly registers the WLUN=
s
> > >> during initialization by calling __scsi_add_device()
> > >>
> > >> ```
> > >> drivers/ufs/core/ufshcd.c line:8137
> > >> /**
> > >>  * ufshcd_scsi_add_wlus - Adds required W-LUs
> > >>  * @hba: per-adapter instance
> > >>  *
> > >>  * UFS device specification requires the UFS devices to support 4 we=
ll known
> > >>  * logical units:
> > >>  *  "REPORT_LUNS" (address: 01h)
> > >>  *  "UFS Device" (address: 50h)
> > >>  *  "RPMB" (address: 44h)
> > >>  *  "BOOT" (address: 30h)
> > >>  * UFS device's power management needs to be controlled by "POWER CO=
NDITION"
> > >>  * field of SSU (START STOP UNIT) command. But this "power condition=
" field
> > >>  * will take effect only when its sent to "UFS device" well known lo=
gical unit
> > >>  * hence we require the scsi_device instance to represent this logic=
al unit in
> > >>  * order for the UFS host driver to send the SSU command for power m=
anagement.
> > >>  *
> > >>  * We also require the scsi_device instance for "RPMB" (Replay Prote=
cted Memory
> > >>  * Block) LU so user space process can control this LU. User space m=
ay also
> > >>  * want to have access to BOOT LU.
> > >>  *
> > >>  * This function adds scsi device instances for each of all well kno=
wn LUs
> > >>  * (except "REPORT LUNS" LU).
> > >>  *
> > >>  * Return: zero on success (all required W-LUs are added successfull=
y),
> > >>  * non-zero error value on failure (if failed to add any of the requ=
ired W-LU).
> > >>  */
> > >> static int ufshcd_scsi_add_wlus(struct ufs_hba *hba)
> > >> {
> > >>     int ret =3D 0;
> > >>     struct scsi_device *sdev_boot, *sdev_rpmb;
> > >>
> > >>     hba->ufs_device_wlun =3D __scsi_add_device(hba->host, 0, 0,
> > >>         ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_UFS_DEVICE_WLUN), NUL=
L);
> > >>     if (IS_ERR(hba->ufs_device_wlun)) {
> > >>         ret =3D PTR_ERR(hba->ufs_device_wlun);
> > >>         hba->ufs_device_wlun =3D NULL;
> > >>         goto out;
> > >>     }
> > >>     scsi_device_put(hba->ufs_device_wlun);
> > >>
> > >>     sdev_rpmb =3D __scsi_add_device(hba->host, 0, 0,
> > >>         ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_RPMB_WLUN), NULL);
> > >>     if (IS_ERR(sdev_rpmb)) {
> > >>         ret =3D PTR_ERR(sdev_rpmb);
> > >>         goto remove_ufs_device_wlun;
> > >>     }
> > >>     ufshcd_blk_pm_runtime_init(sdev_rpmb);
> > >>     scsi_device_put(sdev_rpmb);
> > >>
> > >>     sdev_boot =3D __scsi_add_device(hba->host, 0, 0,
> > >>         ufshcd_upiu_wlun_to_scsi_wlun(UFS_UPIU_BOOT_WLUN), NULL);
> > >>     if (IS_ERR(sdev_boot)) {
> > >>         dev_err(hba->dev, "%s: BOOT WLUN not found\n", __func__);
> > >>     } else {
> > >>         ufshcd_blk_pm_runtime_init(sdev_boot);
> > >>         scsi_device_put(sdev_boot);
> > >>     }
> > >>     goto out;
> > >>
> > >> remove_ufs_device_wlun:
> > >>     scsi_remove_device(hba->ufs_device_wlun);
> > >> out:
> > >>     return ret;
> > >> }
> > >> ```
> > >>
> > >> --------------------------------------------------------------------=
-------
> > >>
> > >> My question is: which approach would be more appropriate for using W=
LUN
> > >> devices in UFS driver on FreeBSD?
> > >>
> > >> (1) Add support in SCSI XPT to probe WLUNs and allocate devices for =
them.
> > >> (2) Follow Linux: have the UFS driver explicitly probe/register WLUN
> > >>     devices itself.
> > >>
> > >> If (2) is preferred, what is the recommended way for the UFSHCI driv=
er to
> > >> probe/register WLUNs in FreeBSD? (Is there an equivalent to Linux=E2=
=80=99s
> > >> __scsi_add_device()?)
> > >
> > >
> > > [[ I didn't trim context here, since it lays out the problem nicely).=
 ]]
> > >
> > > So I can make a case for either (1) or (2).
> > >
> > > It's easy enough to add the enumeration for (1) to scsi_xpt.c It's ju=
st a few more states in the probe state machine (which is a bit complex, bu=
t not terribly so). If we treat them as separate LUNs, then a lot of the in=
frastructure would just work. The key, though, is how do we keep scsi_da, e=
tc from attaching to these new nodes. And would we have generic scsi_foo.c =
drivers for them (so the devices would get their own periph drivers) or whe=
ther there's a higher-level device that would know it could use these devic=
es if it finds them and it wouldn't have a specific specialized kernel driv=
e for them, but instead use a generalized pass thru device.
> > >
> > > I guess a lot of that would depend where things like power management=
 is happening. If it's in the UFS sim, that would suggest a different desig=
n than if this was some super generic thing that would respond to generic e=
vents that maybe the SIMs generate or maybe somebody else.
> > >
> > > If we wanted to go with (1) heavily, we'd need new, generic periph dr=
ivers along with some kind of way to get them to respond to, or report out =
events in the system or with the drive. With only a couple of examples, thi=
s might be difficult to abstract properly.
> > >
> > > If we wanted to go with (2), then we'd likely need some extensions in=
 xpt around this so that the sim could discover, manage and use the WLUN de=
vices.
> > >
> > > Though the more I think about it, maybe it doesn't matter: we want to=
 do the WLUN probing regardless. It's who manages the cam_ed structures aft=
er that. Where does the periph driver live, etc.
> > >
> > > I think I need to go read the UFS specs in more detail than I have ti=
me for this afternoon...
> > >
> > > Warner
> > >
> > > Warner