[PATCH]correctly handling failed opens in usb-serial

Hi,

as recently discussed on lkml the tty layer does call close() if
an open() fails. Usb-serial does not handle this correctly. It does
handle the first open() failing by checking the open count, but if
a second open() fails, the close() will free the resources associated
with the first open.

	Regards
		Oliver
Signed-off-by: Oliver Neukum <[email protected]>
-- 

--- a/drivers/usb/serial/usb-serial.c	2007-03-29 09:47:30.000000000 +0200
+++ b/drivers/usb/serial/usb-serial.c	2007-03-29 09:54:48.000000000 +0200
@@ -59,6 +59,7 @@
 
 static int debug;
 static struct usb_serial *serial_table[SERIAL_TTY_MINORS];	/* initially all NULL */
+static unsigned long error_table[SERIAL_TTY_MINORS / BITS_PER_LONG];
 static spinlock_t table_lock;
 static LIST_HEAD(usb_serial_driver_list);
 
@@ -188,10 +189,12 @@
 	
 	dbg("%s", __FUNCTION__);
 
-	/* get the serial object associated with this tty pointer */
+	/* get the serial object associated with this tty pointer 
+	 * protected by BKL */
 	serial = usb_serial_get_by_index(tty->index);
 	if (!serial) {
 		tty->driver_data = NULL;
+		set_bit(tty->index, error_table);
 		return -ENODEV;
 	}
 
@@ -243,6 +246,7 @@
 	mutex_unlock(&port->mutex);
 bailout_kref_put:
 	usb_serial_put(serial);
+	set_bit(tty->index, error_table);
 	return retval;
 }
 
@@ -250,6 +254,15 @@
 {
 	struct usb_serial_port *port = tty->driver_data;
 
+	/* this handles errors in open
+	 * the tty layer will call close even if open fails
+	 * we need to ignore such calls to close.
+	 * All this happens under BKL - we must not sleep
+	 * before checking for such calls to close
+	 */
+	if (test_and_clear_bit(tty->index, error_table))
+		return;
+
 	if (!port)
 		return;
 
@@ -257,10 +270,7 @@
 
 	mutex_lock(&port->mutex);
 
-	if (port->open_count == 0) {
-		mutex_unlock(&port->mutex);
-		return;
-	}
+	WARN_ON(port->open_count == 0);
 
 	--port->open_count;
 	if (port->open_count == 0) {
@@ -1103,6 +1113,8 @@
 		serial_table[i] = NULL;
 	}
 
+	memset(error_table, 0x0, sizeof(error_table));
+
 	result = bus_register(&usb_serial_bus_type);
 	if (result) {
 		err("%s - registering bus driver failed", __FUNCTION__);


-------------------------------------------------------------------------
Take Surveys. Earn Cash. Influence the Future of IT
Join SourceForge.net's Techsay panel and you'll get the chance to share your
opinions on IT & business topics through brief surveys-and earn cash
http://www.techsay.com/default.php?page=join.php&p=sourceforge&CID=DEVDEV
_______________________________________________
[email protected]
To unsubscribe, use the last form field at:
https://lists.sourceforge.net/lists/listinfo/linux-usb-devel