Synchronous Bulk-Copy into two table

So what about parts records ? they need to copied too !



On May 24, 7:10 pm, Ed Murphy wrote:
> Khafancoder wrote:
> > in my db i have these three tables
>
> > 1.Stores 2.Products 3.Parts
>
> > their structure is something like :
>
> > Stores ----> Products ----> Parts
>
> > Stores
> > ----------------
> > StoreId, StoreName
>
> > Products
> > ----------------
> > ProductId, StoreId, ProductName
>
> > Parts
> > ----------------
> > PartId, ProductId, PartName
>
> > now, in my application i wanna to implement a bulk-copy operation so
> > user can copy products from one store to another one and when a
> > product copied to new store;
>
> > all of it's parts should copy too.
> > in fact i need a method to insert a Product item in Products table and
> > synchronously copy it's parts into Parts table and repeat this steps
> > until all of proucts copied.
>
> > how can i do that without cursors or loops ?
>
> Why do you need to do that at all? It seems like you simply need
> to do the following:
>
> insert into Products n (ProductId, StoreId, ProductName)
> select o.ProductId, @NewStoreId, o.ProductName
> from Products o
> where o.StoreId = @OldStoreId- Hide quoted text -
>
> - Show quoted text -
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Ed Murphy (emurphy42@socal.rr.com) writes:
> Why do you need to do that at all? It seems like you simply need
> to do the following:
>
> insert into Products n (ProductId, StoreId, ProductName)
> select o.ProductId, @NewStoreId, o.ProductName
> from Products o
> where o.StoreId = @OldStoreId

I suspect that Khafancoder's problem may be that ProductId is a
unique key and not a key together with StoreID. The latter may or
may not be a better design depending on the business requirements.

I guess Khafancode will tell us it is not. I hope then he also
gives us more information about his tables: which are the keys,
if there are any IDENTITY column. And also which version of SQL Server
he is using.

--
Erland Sommarskog, SQL Server MVP, esquel.DeleteThis@sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Thanx,
here is the db schema :
http://i12.tinypic.com/4v5qfbb.gif

(PrimaryKey fields are identity too)
(SQL2005)


On May 25, 12:34 am, Erland Sommarskog wrote:
> Ed Murphy (emurph...@socal.rr.com) writes:
> > Why do you need to do that at all? It seems like you simply need
> > to do the following:
>
> > insert into Products n (ProductId, StoreId, ProductName)
> > select o.ProductId, @NewStoreId, o.ProductName
> > from Products o
> > where o.StoreId = @OldStoreId
>
> I suspect that Khafancoder's problem may be that ProductId is a
> unique key and not a key together with StoreID. The latter may or
> may not be a better design depending on the business requirements.
>
> I guess Khafancode will tell us it is not. I hope then he also
> gives us more information about his tables: which are the keys,
> if there are any IDENTITY column. And also which version of SQL Server
> he is using.
>
> --
> Erland Sommarskog, SQL Server MVP, esq....DeleteThis@sommarskog.se
>
> Books Online for SQL Server 2005 athttp://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books...
> Books Online for SQL Server 2000 athttp://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Khafancoder ( ) writes:
> Thanx,
> here is the db schema :
> http://i12.tinypic.com/4v5qfbb.gif
>
> (PrimaryKey fields are identity too)
> (SQL2005)

Permit me to remark that the designe does not look good to me. I don't
see why you would copy products and parts from one store to another,
and give them new ids. If you have a product "Widgets", would it not
be the same product in each store?

Had you not used the IDENTITY property, it would have been an easy thing:

BEGIN TRANSACTION

SELECT @newstoreid = colaesce(MAX(storeid), 0) + 1
FROM Stores WITH (UPDLOCK)

INSERT Stores (StoredId, StoreName)
VALUES(@newstoreid, @newstorename)

SELECT @maxprodid = MAX(ProductId) FROM Products
SELECT @minoldprodid = MIN(ProductId)
FROM Stores
WHERE StoredId = @oldstoreid

INSERT Products (ProductId, StoreId, ProductName, ProductDescription)
SELECT @maxprodid + 1 + @minprodid - ProductId, @newstoreid,
ProductName, ProductDescription
FROM Products
WHERE StoreId = @oldstoreid

SELECT @maxpartid = MAX(PartId) FROM Parts
SELECT @minpartid = MIN(Pa.PartId)
FROM Parts Pa
JOIN Products Pr ON Pa.ProductId = Pr.ProductID
WHERE Pr.StoreID = @oldstoreid

INSERT Parts(PartId, ProductId, Partname)
SELECT @maxpartid + 1 + @minpartid - Pa.PartId,
@maxprodid + 1 + @minprodid - Pr.ProductId, Pa.Partname
FROM Parts Pa
JOIN Products Pr ON Pa.ProductId = Pr.ProductID
WHERE Pr.StoreID = @oldstoreid

COMMIT TRANSACTION

Since you use IDENTITY, things become far more cumbersome, and you are
probably best off changing the design so that you are at least not
using identity at all. (I would also prefer a key in Parts that
has ProductId as the first column.)

You could use SET IDENTITY_INSERT and then use the above, but that
requires permissions that a plan user may not have. And it would really
serve to stress that you are using IDENTITY when you shouldn't.

Had PartName and ProductName been known to unique, you could have used
the new OUTPUT clause in SQL 2005, but since they permit NULL, it
does not seem a good idea to use them.


insert into Products n (ProductId, StoreId, ProductName)
select o.ProductId, @NewStoreId, o.ProductName
from Products o
where o.StoreId = @OldStoreId

--
Erland Sommarskog, SQL Server MVP, esquel RemoveThis @sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Thanks,

> Permit me to remark that the designe does not look good to me. I don't
> see why you would copy products and parts from one store to another,
> and give them new ids. If you have a product "Widgets", would it not
> be the same product in each store?

because copied products or parts info may needed to be changed by each
owner.
so, i will consider disabling identity insertion.

Thanks for answers


On May 26, 12:57 am, Erland Sommarskog wrote:
> Khafancoder ( ) writes:
> > Thanx,
> > here is the db schema :
> >http://i12.tinypic.com/4v5qfbb.gif
>
> > (PrimaryKey fields are identity too)
> > (SQL2005)
>
> Permit me to remark that the designe does not look good to me. I don't
> see why you would copy products and parts from one store to another,
> and give them new ids. If you have a product "Widgets", would it not
> be the same product in each store?
>
> Had you not used the IDENTITY property, it would have been an easy thing:
>
> BEGIN TRANSACTION
>
> SELECT @newstoreid = colaesce(MAX(storeid), 0) + 1
> FROM Stores WITH (UPDLOCK)
>
> INSERT Stores (StoredId, StoreName)
> VALUES(@newstoreid, @newstorename)
>
> SELECT @maxprodid = MAX(ProductId) FROM Products
> SELECT @minoldprodid = MIN(ProductId)
> FROM Stores
> WHERE StoredId = @oldstoreid
>
> INSERT Products (ProductId, StoreId, ProductName, ProductDescription)
> SELECT @maxprodid + 1 + @minprodid - ProductId, @newstoreid,
> ProductName, ProductDescription
> FROM Products
> WHERE StoreId = @oldstoreid
>
> SELECT @maxpartid = MAX(PartId) FROM Parts
> SELECT @minpartid = MIN(Pa.PartId)
> FROM Parts Pa
> JOIN Products Pr ON Pa.ProductId = Pr.ProductID
> WHERE Pr.StoreID = @oldstoreid
>
> INSERT Parts(PartId, ProductId, Partname)
> SELECT @maxpartid + 1 + @minpartid - Pa.PartId,
> @maxprodid + 1 + @minprodid - Pr.ProductId, Pa.Partname
> FROM Parts Pa
> JOIN Products Pr ON Pa.ProductId = Pr.ProductID
> WHERE Pr.StoreID = @oldstoreid
>
> COMMIT TRANSACTION
>
> Since you use IDENTITY, things become far more cumbersome, and you are
> probably best off changing the design so that you are at least not
> using identity at all. (I would also prefer a key in Parts that
> has ProductId as the first column.)
>
> You could use SET IDENTITY_INSERT and then use the above, but that
> requires permissions that a plan user may not have. And it would really
> serve to stress that you are using IDENTITY when you shouldn't.
>
> Had PartName and ProductName been known to unique, you could have used
> the new OUTPUT clause in SQL 2005, but since they permit NULL, it
> does not seem a good idea to use them.
>
> insert into Products n (ProductId, StoreId, ProductName)
> select o.ProductId, @NewStoreId, o.ProductName
> from Products o
> where o.StoreId = @OldStoreId
>
> --
> Erland Sommarskog, SQL Server MVP, esq....TakeThisOut@sommarskog.se
>
> Books Online for SQL Server 2005 athttp://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books...
> Books Online for SQL Server 2000 athttp://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

> But of course, I don't know what your real case is like. Maybe you are
> only using stores, products and parts for the sake of the example?

exactly!
so i can't change the design and also i can't disable identity
insertion !!

i'm trying to take an alternative way, something lik this :

--disable check constrains

INSERT INTO Parts
(ProductId, PartName, PartDesc)
SELECT
dbo.CopyProduct(ProductId, @DestinationStoreId) , PartName, PartDesc
FROM Parts WHERE StoreId=@StoreId

--enable check constrains

and CopyProduct functions is supposed to copy requested ProductItem
and return it's Id,
but i can't do that in UDF !!

CREATE FUNCTION CopyProduct(@ProductId bigint, @DestinationStoreId
bigint) RETURNS bigint
AS
BEGIN

INSERT INTO Products (StoreId, ProductName)
SELECT @DestinationStoreId, ProductName FROM Products WHERE
ProductId=@ProductId

RETURN SCOPE_IDENTITY()
END


i think it should solve the problem, but because of sqlserver
restriction i can't do that in a function !!

any idea ?






On May 26, 11:50 am, Erland Sommarskog wrote:
> Khafancoder ( ) writes:
> > because copied products or parts info may needed to be changed by each
> > owner.
> > so, i will consider disabling identity insertion.
>
> I still don't think this is a good design. Reasonably, there are
> inherit properties with a product that does not depend on the store.
>
> I think you need a new table ProductStores:
>
> CREATE TABLE StoreProducts(StoreId bigint NOT NULL,
> ProductId bigint NOT NULL,
> CONSTRAINT pk_ProductStores PRIMARY KEY(StoreId,ProductId))
>
> This table can then be augmented with columns that the store owner can
> set as he pleases. If needed, you could also have a StoreProductParts,
> with (StoreId, ProductId, PartNo) as key and with (StoreId, ProductId)
> as foreign key to StoreProducts and (ProductId, PartNo) as foreign key
> to Parts. But it seems funny to me that the same product would have
> different parts in different stores.
>
> But of course, I don't know what your real case is like. Maybe you are
> only using stores, products and parts for the sake of the example?
>
> --
> Erland Sommarskog, SQL Server MVP, esq... RemoveThis @sommarskog.se
>
> Books Online for SQL Server 2005 athttp://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books...
> Books Online for SQL Server 2000 athttp://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Khafancoder ( ) writes:
> because copied products or parts info may needed to be changed by each
> owner.
> so, i will consider disabling identity insertion.

I still don't think this is a good design. Reasonably, there are
inherit properties with a product that does not depend on the store.

I think you need a new table ProductStores:

CREATE TABLE StoreProducts(StoreId bigint NOT NULL,
ProductId bigint NOT NULL,
CONSTRAINT pk_ProductStores PRIMARY KEY(StoreId,ProductId))

This table can then be augmented with columns that the store owner can
set as he pleases. If needed, you could also have a StoreProductParts,
with (StoreId, ProductId, PartNo) as key and with (StoreId, ProductId)
as foreign key to StoreProducts and (ProductId, PartNo) as foreign key
to Parts. But it seems funny to me that the same product would have
different parts in different stores.

But of course, I don't know what your real case is like. Maybe you are
only using stores, products and parts for the sake of the example?

--
Erland Sommarskog, SQL Server MVP, esquel.RemoveThis@sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Khafancoder ( ) writes:
>> But of course, I don't know what your real case is like. Maybe you are
>> only using stores, products and parts for the sake of the example?
>
> exactly!
> so i can't change the design and also i can't disable identity
> insertion !!

I presume then that the real tables are more complex than the mock-up
posted.

There is one final question, I will have to ask: in the real Products
table are there any columns beside the IDENTITY column that are unique
within a store? If the answer is yes, then my answer is yes, you
can do it set-based, and if you tell which version of SQL Server you
are using, I can sketch a solution.

If the answer is no, you will have to run a loop and insert the
products one-by-one. You should still be able to copy all parts for
a product in one go, as long as you don't need the part it anywhere.
If this sounds clunky to you, it is because the design is not optimal.

> and CopyProduct functions is supposed to copy requested ProductItem
> and return it's Id,
> but i can't do that in UDF !!
>
> CREATE FUNCTION CopyProduct(@ProductId bigint, @DestinationStoreId
> bigint) RETURNS bigint
> AS
> BEGIN
>
> INSERT INTO Products (StoreId, ProductName)
> SELECT @DestinationStoreId, ProductName FROM Products WHERE
> ProductId=@ProductId
>
> RETURN SCOPE_IDENTITY()
> END

Right. A user-defined function cannot change database state, so
that's a non-starter.




--
Erland Sommarskog, SQL Server MVP, esquel DeleteThis @sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

> There is one final question, I will have to ask: in the real Products
> table are there any columns beside the IDENTITY column that are unique
> within a store? If the answer is yes, then my answer is yes, you
> can do it set-based, and if you tell which version of SQL Server you
> are using, I can sketch a solution.

no, there isn't.
but isn't possible through StoreId & ProductId together ?

and finally, isn't any other alternative to do insert in UDFs ?
or executing an sp in a select statement ?

Thanks



On May 26, 11:26 pm, Erland Sommarskog wrote:
> Khafancoder ( ) writes:
> >> But of course, I don't know what your real case is like. Maybe you are
> >> only using stores, products and parts for the sake of the example?
>
> > exactly!
> > so i can't change the design and also i can't disable identity
> > insertion !!
>
> I presume then that the real tables are more complex than the mock-up
> posted.
>
> There is one final question, I will have to ask: in the real Products
> table are there any columns beside the IDENTITY column that are unique
> within a store? If the answer is yes, then my answer is yes, you
> can do it set-based, and if you tell which version of SQL Server you
> are using, I can sketch a solution.
>
> If the answer is no, you will have to run a loop and insert the
> products one-by-one. You should still be able to copy all parts for
> a product in one go, as long as you don't need the part it anywhere.
> If this sounds clunky to you, it is because the design is not optimal.
>
> > and CopyProduct functions is supposed to copy requested ProductItem
> > and return it's Id,
> > but i can't do that in UDF !!
>
> > CREATE FUNCTION CopyProduct(@ProductId bigint, @DestinationStoreId
> > bigint) RETURNS bigint
> > AS
> > BEGIN
>
> > INSERT INTO Products (StoreId, ProductName)
> > SELECT @DestinationStoreId, ProductName FROM Products WHERE
> > ProductId=@ProductId
>
> > RETURN SCOPE_IDENTITY()
> > END
>
> Right. A user-defined function cannot change database state, so
> that's a non-starter.
>
> --
> Erland Sommarskog, SQL Server MVP, esq....TakeThisOut@sommarskog.se
>
> Books Online for SQL Server 2005 athttp://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books...
> Books Online for SQL Server 2000 athttp://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Khafancoder ( ) writes:
>> There is one final question, I will have to ask: in the real Products
>> table are there any columns beside the IDENTITY column that are unique
>> within a store? If the answer is yes, then my answer is yes, you
>> can do it set-based, and if you tell which version of SQL Server you
>> are using, I can sketch a solution.
>
> no, there isn't.
> but isn't possible through StoreId & ProductId together ?

I don't know what you have in mind here. The key problem is that
when you insert many rows into a table with an IDENTITY column, and you
need to know the generated IDENTITY value for each row and there is
no other columns that identifies the rows, you will need to insert the
rows one by one. Or override the IDENTITY-generation with SET
IDENTITY_INSERT ON. But the latter that the user has privilege for this
operation; it cannot be granted through ownership chain.

> and finally, isn't any other alternative to do insert in UDFs ?
> or executing an sp in a select statement ?

It's not really meaningful of talking about an alternative to something
which is completely dead.

What you really should do is to change the database design, because the
current design is the root to your problem.

--
Erland Sommarskog, SQL Server MVP, esquel DeleteThis @sommarskog.se

Books Online for SQL Server 2005 at
http://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books.mspx
Books Online for SQL Server 2000 at
http://www.microsoft.com/sql/prodinfo/previousversions/books.mspx
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Hi again !

i finally decided to disable identity insertion and do the copy
operation by using temporary Map tables which maps Old Ids and New Ids
so :
--------------------------------------------------------------------
CREATE TABLE #MapProducts (SourceProductId bigint, DestProductId
bigint)

INSERT INTO #MapProducts (SourceProductId, DestProductId)
SELECT ProductId, CASE WHEN
((SELECT COUNT(*) FROM #MapProducts) > 0) THEN (SELECT
MAX(DestProductId) + 1 FROM #MapProducts)
ELSE (SELECT MAX(ProductId) + 1 FROM Products) END
FROM Products WHERE StoreId=@SourceStoreId
--------------------------------------------------------------------


but another problem, this line :
--------------------------------------------------------------------
CASE WHEN ((SELECT COUNT(*) FROM #MapProducts) > 0) THEN (SELECT
MAX(DestProductId) + 1 FROM #MapProducts)
--------------------------------------------------------------------
won't be executed because sql engine calculate COUNT before do the
insert operation.

how could i solve that ?
is it possible to force INSERT command to calculate COUNT after
inserting *each record* ?

Thanks
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Any Idea ?
 >> Stay informed about: Synchronous Bulk-Copy into two table 

Thanks for reply,

i think it won't be a right logic for creating new id s ; or may be i
didn't understand the algorithm correctly


> SELECT @maxprodid = MAX(ProductId) FROM Products
> SELECT @minoldprodid = MIN(ProductId)
> FROM Stores
> WHERE StoredId = @oldstoreid
>
> INSERT Products (ProductId, StoreId, ProductName, ProductDescription)
> SELECT @maxprodid + 1 + @minprodid - ProductId, @newstoreid,
> ProductName, ProductDescription
> FROM Products
> WHERE StoreId = @oldstoreid


Suppose Product Ids are from 1 to 100

so
@maxprodid=100
@minoldprodid = 1

and so, if we gonna to copy a record with id=50 it will be :

@maxprodid + 1 + @minprodid - ProductId = 100 + 1 + 1 - 50
= 52

and 52 is id of an existing product record

Thanks in Advance




On May 31, 12:47 am, Erland Sommarskog wrote:
> Khafancoder ( ) writes:
> > i finally decided to disable identity insertion and do the copy
> > operation by using temporary Map tables which maps Old Ids and New Ids
> > so :
> > --------------------------------------------------------------------
> > CREATE TABLE #MapProducts (SourceProductId bigint, DestProductId
> > bigint)
>
> > INSERT INTO #MapProducts (SourceProductId, DestProductId)
> > SELECT ProductId, CASE WHEN
> > ((SELECT COUNT(*) FROM #MapProducts) > 0) THEN (SELECT
> > MAX(DestProductId) + 1 FROM #MapProducts)
> > ELSE (SELECT MAX(ProductId) + 1 FROM Products) END
> > FROM Products WHERE StoreId=@SourceStoreId
> > --------------------------------------------------------------------
>
> > but another problem, this line :
> > --------------------------------------------------------------------
> > CASE WHEN ((SELECT COUNT(*) FROM #MapProducts) > 0) THEN (SELECT
> > MAX(DestProductId) + 1 FROM #MapProducts)
> > --------------------------------------------------------------------
> > won't be executed because sql engine calculate COUNT before do the
> > insert operation.
>
> > how could i solve that ?
> > is it possible to force INSERT command to calculate COUNT after
> > inserting *each record* ?
>
> That won't fly, as you have noticed. You need to compute the new
> id:s from what is given before you start inserting. And this should
> not be impossible. Please review this piece of code, which is the
> same I posted a couple of days back:
>
> BEGIN TRANSACTION
>
> SELECT @newstoreid = colaesce(MAX(storeid), 0) + 1
> FROM Stores WITH (UPDLOCK)
>
> INSERT Stores (StoredId, StoreName)
> VALUES(@newstoreid, @newstorename)
>
> SELECT @maxprodid = MAX(ProductId) FROM Products
> SELECT @minoldprodid = MIN(ProductId)
> FROM Stores
> WHERE StoredId = @oldstoreid
>
> INSERT Products (ProductId, StoreId, ProductName, ProductDescription)
> SELECT @maxprodid + 1 + @minprodid - ProductId, @newstoreid,
> ProductName, ProductDescription
> FROM Products
> WHERE StoreId = @oldstoreid
>
> SELECT @maxpartid = MAX(PartId) FROM Parts
> SELECT @minpartid = MIN(Pa.PartId)
> FROM Parts Pa
> JOIN Products Pr ON Pa.ProductId = Pr.ProductID
> WHERE Pr.StoreID = @oldstoreid
>
> INSERT Parts(PartId, ProductId, Partname)
> SELECT @maxpartid + 1 + @minpartid - Pa.PartId,
> @maxprodid + 1 + @minprodid - Pr.ProductId, Pa.Partname
> FROM Parts Pa
> JOIN Products Pr ON Pa.ProductId = Pr.ProductID
> WHERE Pr.StoreID = @oldstoreid
>
> COMMIT TRANSACTION
>
> --
> Erland Sommarskog, SQL Server MVP, esq....DeleteThis@sommarskog.se
>
> Books Online for SQL Server 2005 athttp://www.microsoft.com/technet/prodtechnol/sql/2005/downloads/books...
> Books Online for SQL Server 2000 athttp://www.microsoft.com/sql/prodinfo/previousversions/books.mspx- Hide quoted text -
>
> - Show quoted text -
 >> Stay informed about: Synchronous Bulk-Copy into two table